Bayesian Analysis in Psychology

Why use the Bayesian Framework?

In short, because it’s:

• Better
• Simpler
• Superior
• Preferable
• More appropriate
• More desirable
• More useful
• More valuable

What is the Bayesian Framework?

Once we agreed that the Bayesian framework is the right way to go, you might wonder what is the Bayesian framework. What’s all the fuss about?

Omitting the maths behind it, let’s just say that:

• The frequentist guy tries to estimate “the real effect”. The “real” value of the correlation between X and Y. It returns a “point-estimate” (i.e., a single value) of the “real” correlation (e.g., r = 0.42), considering that the data is sampled at random from a “parent”, usually normal distribution of data.
• The Bayesian master assumes no such thing. The data are what they are. Based on this observed data (and eventually from its expectations), the Bayesian sampling algorithm will return a probability distribution of the effect that is compatible with the observed data. For the correlation between X and Y, it will return a distribution that says “the most probable effect is 0.42, but this data is also compatible with correlations of 0.12 or 0.74”.
• To characterize our effects, no need of p values or other mysterious indices. We simply describe the posterior distribution (i.e., the distribution of the effect). We can present the median (better than the mean, as it actually means that the effect has 50% of chance of being higher and 50% of chance of being lower), the MAD (a median-based, robust equivalent of SD) and other stuff such as the 90% HDI, called here credible interval.

The “Posterior” distribution

Let’s imagine two numeric variables, Y and X. The correlation between them is r = -0.063 (p < .05). A Bayesian analysis would return the probability of distribution of this effect (the posterior), that we can characterize using several indices (centrality (median or mean), dispersion (SD or Median Absolute Deviation - MAD), etc.). Let’s plot the posterior distribution of the possible correlation values that are compatible with our data.

Posterior probability distribution of the correlation between X and Y

In this example (based on real data):

• The median of the posterior distribution is really close to the r coefficient obtained through a “normal” correlation analysis, appearing as a good point-estimate of the correlation. Moreover, and unlike the frequentist estimate, the median has an intuitive meaning (as it cuts the distribution in two equal parts): the “true” effect has 50% of chance of being higher and 50% of chance of being lower.
• We can also compute the 90% credible interval, which shows where the true effect can fall with a probability of 90% (better than 95% CI, see Kruschke, 2015).
• Finally, we can also compute the MPE, i.e., the maximum probability of effect, which is the probability that the effect is in the median’s direction (i.e., negative in this example): the area in red. It interesting to note that the probability that the effect is opposite ((100 - MPE) / 100, here (100-98.85) / 100 = 0.011) is relatively close to the actual p value obtained through frequentist correlation (p = 0.025).

Now that you’re familiar with posterior distributions, the core difference of the Bayesian framework, let’s practice!

Model Exploration

# Let's fit our model
fit <- rstanarm::stan_glm(Life_Satisfaction ~ Tolerating, data=df)

Let’s check the results:

# Format the results using analyze()
results <- psycho::analyze(fit)

# We can extract a formatted summary table
summary(results, round = 2)

For each parameter of the model, the summary shows:

• the median of the posterior distribution of the parameter (can be used as a point estimate, similar to the beta of frequentist models).
• the Median Absolute Deviation (MAD), a robust measure of dispertion (could be seen as a robust version of SD).
• the Credible Interval (CI) (by default, the 90% CI; see Kruschke, 2018), representing a range of possible parameter values.
• the Maximum Probability of Effect (MPE), the probability that the effect is positive or negative (depending on the median’s direction).
• the Overlap (O), the percentage of overlap between the posterior distribution and a normal distribution of mean 0 and same SD than the posterior. Can be interpreted as the probability that a value from the posterior distribution comes from a null distribution of same uncertainty (width).

It also returns the (unadjusted) R2 (which represents the percentage of variance of the outcome explained by the model). In the Bayesian framework, the R2 is also estimated with probabilities. As such, characteristics of its posterior distribution are returned.

We can also print a formatted version:

print(results)

## We fitted a Markov Chain Monte Carlo gaussian (link = identity) model (4 chains, each with iter = 2000; warmup = 1000; thin = 1; post-warmup = 1000) to predict Life_Satisfaction (formula = Life_Satisfaction ~ Tolerating). The model's priors were set as follows: 
## 
##   ~ normal (location = (0), scale = (3.11))
## 
## 
## The model explains about 2.35% of the outcome's variance (MAD = 0.0085, 90% CI [0.0098, 0.037], Adj. R2 = 0.020). The intercept is at 4.06 (MAD = 0.15, 90% CI [3.78, 4.29]). Within this model:
## 
##   - The effect of Tolerating has a probability of 100% of being positive (Median = 0.19, MAD = 0.035, 90% CI [0.13, 0.25], O = 0.86%). It can be considered as small or very small with respective probabilities of 4.75% and 95.25%.

Note that the print() returns also additional info, such as the 100% CI of the R2 and, for each parameter, the limits of the range defined by the MPE.

Interpretation

For now, omit the part dedicated to priors. We’ll see it in the next chapters. Let’s rather interpret the part related to effects.

Full Bayesian mixed linear models are fitted using the rstanarm R wrapper for the stan probabilistic language (Gabry & Goodrich, 2016). Bayesian inference was done using Markov Chain Monte Carlo (MCMC) sampling. The prior distributions of all effects were set as weakly informative (mean = 0, SD = 3.11), meaning that we did not expect effects different from null in any particular direction. For each model and each coefficient, we will present several characteristics of the posterior distribution, such as its median (a robust estimate comparable to the beta from frequentist linear models), MAD (median absolute deviation, a robust equivalent of standard deviation) and the 90% credible interval. Instead of the p value as an index of effect existence, we also computed the maximum probability of effect (MPE), i.e., the maximum probability that the effect is different from 0 in the median’s direction. For our analyses, we will consider an effect as inconsistent (i.e., not probable enough) if its MPE is lower than 90% (however, beware not to fall in a p value-like obsession).

The current model explains about 2.35% of life satisfaction variance. Within this model, a positive linear relationship between life satisfaction and tolerating exists with high probability (Median = 0.19, MAD = 0.035, 90% CI [0.13, 0.25], MPE = 100%).

Model Visualization

To visualize the model, the most neat way is to extract a “reference grid” (i.e., a theorethical dataframe with balanced data).

refgrid <- df %>% 
  select(Tolerating) %>% 
  psycho::refdata(length.out=10)

predicted <- psycho::get_predicted(fit, newdata=refgrid)

predicted

Our refgrid is made of equally spaced (balanced) predictor values. It also include the median of the posterior prediction, as well as 90% credible intervals. Now, we can plot it as follows:

ggplot(predicted, aes(x=Tolerating, y=Life_Satisfaction_Median)) +
  geom_line() +
  geom_ribbon(aes(ymin=Life_Satisfaction_CI_5, 
                  ymax=Life_Satisfaction_CI_95), 
              alpha=0.1)

Model Exploration

# Let's fit our model
fit <- rstanarm::stan_glm(Life_Satisfaction ~ Salary, data=df)

Let’s check the results:

# Format the results using analyze()
results <- psycho::analyze(fit)

# We can extract a formatted summary table
print(results)

## We fitted a Markov Chain Monte Carlo gaussian (link = identity) model (4 chains, each with iter = 2000; warmup = 1000; thin = 1; post-warmup = 1000) to predict Life_Satisfaction (formula = Life_Satisfaction ~ Salary). The model's priors were set as follows: 
## 
##   ~ normal (location = (0, 0), scale = (3.61, 3.61))
## 
## 
## The model explains about 0.43% of the outcome's variance (MAD = 0.0039, 90% CI [0, 0.011], Adj. R2 = -0.0037). The intercept is at 4.76 (MAD = 0.063, 90% CI [4.67, 4.87]). Within this model:
## 
##   - The effect of Salary<2000 has a probability of 86.30% of being positive (Median = 0.13, MAD = 0.11, 90% CI [-0.072, 0.31], O = 58.97%). It can be considered as very small with a probability of 86.30%.
##   - The effect of Salary2000+ has a probability of 91.90% of being positive (Median = 0.20, MAD = 0.15, 90% CI [-0.026, 0.44], O = 49.23%). It can be considered as very small with a probability of 91.90%.

Post-hoc / Contrasts / Comparisons

What interest us is the pairwise comparison between the groups. The get_contrasts function computes the estimated marginal means (least-squares means), i.e., the means of each group estimated by the model, as well as the contrasts.

contrasts <- psycho::get_contrasts(fit, "Salary")

We can see the estimated means like that:

contrasts$means

And the contrasts comparisons like that:

contrasts$contrasts

As we can see, the only probable difference (MPE > 90%) is between Salary <1000 and Salary 2000+.

Model Visualization

ggplot(contrasts$means, aes(x=Level, y=Median, group=1)) +
  geom_line() +
  geom_pointrange(aes(ymin=CI_lower, ymax=CI_higher)) +
  ylab("Life Satisfaction") +
  xlab("Salary")

Model Exploration

# Let's fit our model
fit <- rstanarm::stan_glm(Sex ~ Adjusting, data=df, family = "binomial")

First, let’s check our model:

# Format the results using analyze()
results <- psycho::analyze(fit)

# We can extract a formatted summary table
summary(results, round = 2)

It appears that the link between adjusting and the sex is highly probable (MPE > 90%). But in what direction? To know that, we have to find out what is the intercept (the reference level).

## [1] "F" "M"

As female is the first level, it means that it is the intercept. Based on our model, an increase of 1 on the scale of adjusting will increase the probability (expressed in log odds ratios) of being a male.

Model Visualization

To visualize this type of model, we have to derive a reference grid.

refgrid <- df %>% 
  select(Adjusting) %>% 
  psycho::refdata(length.out=10)
  
predicted <- psycho::get_predicted(fit, newdata=refgrid)

Note that get_predicted automatically transformed log odds ratios (the values in which the model is expressed) to probabilities, easier to apprehend.

ggplot(predicted, aes(x=Adjusting, y=Sex_Median)) +
  geom_line() +
  geom_ribbon(aes(ymin=Sex_CI_5, 
                  ymax=Sex_CI_95), 
              alpha=0.1) +
  ylab("Probability of being a male")

We can nicely see the non-linear relationship between adjusting and the probability of being a male.

Model Exploration

# Let's fit our model
fit <- rstanarm::stan_glm(Life_Satisfaction ~ Concealing * Sex, data=df)

Let’s check our model:

# Format the results using analyze()
results <- psycho::analyze(fit)

# We can extract a formatted summary table
summary(results, round = 2)

Again, it is important to notice that the intercept (the baseline) corresponds here to Concealing = 0 and Sex = F. As we can see next, there is, with high probability, a negative linear relationship between concealing (for females only) and life satisfaction. Also, at the (theorethical) intercept (when concealing = 0), the males have a lower life satisfaction. Finally, the interaction is also probable. This means that when the participant is a male, the relationship between concealing and life satisfaction is significantly different (increased by 0.17. In other words, we could say that the relationship is of -0.10+0.17=0.07 in men).

Model Visualization

How to represent this type of models? Again, we have to generate a reference grid.

refgrid <- df %>% 
  select(Concealing, Sex) %>% 
  psycho::refdata(length.out=10)

predicted <- psycho::get_predicted(fit, newdata=refgrid)
predicted

As we can see, the reference grid is balanced in terms of factors and numeric predictors. Now, to plot this becomes very easy!

ggplot(predicted, aes(x=Concealing, y=Life_Satisfaction_Median, fill=Sex)) +
  geom_line(aes(colour=Sex)) +
  geom_ribbon(aes(fill=Sex,
                  ymin=Life_Satisfaction_CI_5, 
                  ymax=Life_Satisfaction_CI_95), 
              alpha=0.1) +
  ylab("Life Satisfaction")

We can see that the error for the males is larger, due to less observations.

Why use mixed-models?

• From Makowski et al. (under review):

The Mixed modelling framework allows estimated effects to vary by group at lower levels while estimating population-level effects through the specification of fixed (explanatory variables) and random (variance components) effects. Outperforming traditional procedures such as repeated measures ANOVA (Kristensen & Hansen, 2004), these models are particularly suited to cases in which experimental stimuli are heterogeneous (e.g., images) as the item-related variance, in addition to the variance induced by participants, can be accounted for (Baayen, Davidson, & Bates, 2008; Magezi, 2015). Moreover, mixed models can handle unbalanced data, nested designs, crossed random effects and missing data.

As for how to run this type of analyses, it is quite easy. Indeed, all what has been said previously remains the same for mixed models. Except that there are random effects (specified by putting + (1|random_term) in the formula). For example, we might want to consider the salary as a random effect (to “adjust” (so to speak) for the fact that the data is structured in two groups). Let’s explore the relationship between the tendency to conceal emotions and age (adjusted for salary).

Model Exploration

# Let's fit our model (it takes more time)
fit <- rstanarm::stan_lmer(Concealing ~ Age + (1|Salary), data=df)

Let’s check our model:

# Format the results using analyze()
results <- psycho::analyze(fit)

# We can extract a formatted summary table
summary(results, round = 2)

As we can see, the linear relationship has only a moderate probability of being different from 0.

Model Visualization

refgrid <- df %>% 
  select(Age) %>% 
  psycho::refdata(length.out=10)

# We name the predicted dataframe by adding '_linear' to keep it for further comparison (see next part)
predicted_linear <- psycho::get_predicted(fit, newdata=refgrid)

ggplot(predicted_linear, aes(x=Age, y=Concealing_Median)) +
  geom_line() +
  geom_ribbon(aes(ymin=Concealing_CI_5, 
                  ymax=Concealing_CI_95), 
              alpha=0.1)

Model Exploration

# Let's fit our model (it takes more time)
fit <- rstanarm::stan_lmer(Concealing ~ poly(Age, 2, raw=TRUE) + (1|Salary), data=df)

Let’s check our model:

# Format the results using analyze()
results <- psycho::analyze(fit)

# We can extract a formatted summary table
summary(results, round = 2)

As we can see, both the linear relationship and the second order curvature are highly probable. However, when setting raw=TRUE in the formula, the coefficients become unintepretable. So let’s visualize them.

Model Visualization

The model visualization routine is similar to the previous ones.

refgrid <- df %>% 
  select(Age) %>% 
  psycho::refdata(length.out=20)

predicted_poly <- psycho::get_predicted(fit, newdata=refgrid)

ggplot(predicted_poly, aes(x=Age, y=Concealing_Median)) +
  geom_line() +
  geom_ribbon(aes(ymin=Concealing_CI_5, 
                  ymax=Concealing_CI_95), 
              alpha=0.1)

As we can see, adding the polynomial degree changes the relationship. Since the model is here very simple, we can add on the plot the actual points (however, they do not take into account the random effects and such), as well as plot the two models. Also, let’s make it “dynamic” using plotly.

p <- ggplot() +
  # Linear model
  geom_line(data=predicted_linear, 
            aes(x=Age, y=Concealing_Median),
            colour="blue",
            size=1) +
  geom_ribbon(data=predicted_linear, 
              aes(x=Age,
                  ymin=Concealing_CI_5,
                  ymax=Concealing_CI_95), 
              alpha=0.1,
              fill="blue") +
  # Polynormial Model
  geom_line(data=predicted_poly, 
            aes(x=Age, y=Concealing_Median),
            colour="red",
            size=1) +
  geom_ribbon(data=predicted_poly, 
              aes(x=Age,
                  ymin=Concealing_CI_5, 
                  ymax=Concealing_CI_95), 
              fill="red",
              alpha=0.1) +
  # Actual data
  geom_point(data=df, aes(x=Age, y=Concealing))

library(plotly) # To create interactive plots
ggplotly(p) # To transform a ggplot into an interactive plot

It’s good to take a few steps back and look at the bigger picture :)

Weakly informative priors

As you might have notice, we didn’t specify any priors in the previous analyses. In fact, we let the algorithm define and set weakly informative priors, designed to provide moderate regularization and help stabilize computation, without biasing the effect direction. For example, a wealky informative prior, for a standardized predictor (with mean = 0 and SD = 1) could be a normal distribution with mean = 0 and SD = 1. This means that the effect of this predictor is expected to be equally probable in any direction (as the distribution is symmetric around 0), with probability being higher close to 0 and lower far from 0.

While this prior doesn’t bias the direction of the Bayesian (MCMC) sampling, it suggests that having an effect of 100 (i.e., located at 100 SD of the mean as our variables are standardized) is highly unprobable, and that an effect close to 0 is more probable.

To better play with priors, let’s start by standardizing our dataframe.

# Standardize (scale and center) the numeric variables
dfZ <- psycho::standardize(df)

Then, we can explicitly specify a weakly informative prior for all effects of the model.

# Let's fit our model
fit <- rstanarm::stan_glm(Life_Satisfaction ~ Tolerating, 
                          data=dfZ,
                          prior=normal(location = 0, # Mean
                                       scale = 1, # SD
                                       autoscale=FALSE)) # Don't adjust scale automatically

Let’s plot the prior (the expectation) against the posterior (the estimated effect) distribution.

results <- psycho::analyze(fit)

# Extract the posterior
posterior <- results$values$effects$Tolerating$posterior

# Create a posterior with the prior and posterior distribution and plot them.
data.frame(posterior = posterior,
           prior = rnorm(length(posterior), 0, 1)) %>% 
  ggplot() +
  geom_density(aes(x=posterior), fill="lightblue", alpha=0.5) +
  geom_density(aes(x=prior), fill="blue", alpha=0.5) +
  scale_y_sqrt() # Change the Y axis so the plot is less ugly

This plot is rather ugly, because our posterior is very precise (due to the large sample) compared to the prior.

Informative priors

Although the default priors tend to work well, prudent use of more informative priors is encouraged. It is important to underline that setting informative priors (if realistic), does not overbias the analysis. In other words, is only “directs” the sampling: if the data are highly informative about the parameter values (enough to overwhelm the prior), a prudent informative prior (even if oppositive to the observed effect) will yield similar results to a non-informative prior. In other words, you can’t change the results dramatically by tweaking the priors. But as the amount of data and/or the signal-to-noise ratio decrease, using a more informative prior becomes increasingly important. Of course, if you see someone using a prior with mean = 42 and SD = 0.0001, you should look at his results with caution…

Anyway, see the official rstanarm documentation for details.

Plot all iterations

As Bayesian models usually generate a lot of samples (iterations), one could want to plot them as well, instead (or along) the posterior “summary”. This can be done quite easily by extracting all the iterations in get_predicted.

# Fit the model
fit <- rstanarm::stan_glm(Sex ~ Adjusting, data=df, family = "binomial")

# Generate a new refgrid
refgrid <- df %>% 
  select(Adjusting) %>% 
  psycho::refdata(length.out=10)

# Get predictions and keep iterations
predicted <- psycho::get_predicted(fit, newdata=refgrid, keep_iterations=TRUE)

# Reshape this dataframe to have iterations as factor
predicted <- predicted %>% 
  tidyr::gather(Iteration, Iteration_Value, starts_with("iter"))

# Plot iterations as well as the median prediction
ggplot(predicted, aes(x=Adjusting)) +
  geom_line(aes(y=Iteration_Value, group=Iteration), size=0.3, alpha=0.01) +
  geom_line(aes(y=Sex_Median), size=1) + 
  ylab("Male Probability\n")