This worksheet builds on three previous worksheets: within-subject differences, understanding interactions, and factorial differences. We therefore assume you are familiar with the concepts and commands in those worksheets. If that is not the case, read (or re-read) those worksheets before proceeding.

In the factorial differences worksheet, we did a two-factor Bayesian ANOVA where one factor was within-subject and the other was between-subjects. In this worksheet, we’ll use the same data frame to illustrate another couple of two-factor ANOVAs.

Return to the project that you used for the factorial differences worksheet, which you named
**rminr-data**.

Ensure you have the files required for this worksheet, by asking git
to “`pull`

” the repository. Select the `Git`

tab,
which is located in the row of tabs which includes the
`Environment`

tab. Click the `Pull`

button with a
downward pointing arrow. A window will open showing the files which have
been pulled from the repository. Close the `Git pull`

window.

Now, open a new blank script and save it as
**anova4.R**. Put all comments and commands in that script,
and save the script regularly.

First, you’ll need to load some standard packages, load the data we used in the previous worksheet, and set the appropriate columns up as factors.

**Enter these comments and commands into your script, and run
them:**

```
# Clear workspace
rm(list = ls())
# Load packages
library(tidyverse)
library(BayesFactor, quietly = TRUE)
# Load data
words <- read_csv("wordnaming2.csv")
# Set columns as factors
words$sex <- factor(words$sex)
words$subj <- factor(words$subj)
words$medit <- factor(words$medit)
words$congru <- factor(words$congru)
words$block <- factor(words$block)
```

We can use `anovaBF`

to do a factorial Bayesian ANOVA
where both factors are between-subjects. It works exactly the same way
as before. For example, we can do a factorial Bayesian ANOVA for
`sex`

and `medit`

like this:

First we create a subject-level summary of the data - this involves
averaging across the 90 data points that contribute to the reaction time
(RT) measure for each participant. We do this using the now-familiar
`group_by`

and `summarise`

commands.

**Enter this comment and commands into your script, and run
them:**

```
# Create subject-level summary
wordsum <- words %>% group_by(subj, medit, sex) %>% summarise(rt = mean(rt))
```

Now we run the Bayesian ANOVA, as follows; **enter these
commands into your script, and run them:**

```
# Calculate Bayes Factors
bf2bs <- anovaBF(
formula = rt ~ medit*sex,
data = data.frame(wordsum))
```

Note that, unlike the Bayesian ANOVAs we previously did that included
a within-subjects factor, we do not include `+ subj`

or
`whichRandom = 'subj'`

in this entirely between-subjects
ANOVA. This is because we have created a subject-level summary of the
data in which each participant only contributes one data point.

The output is interpreted the same way as before; **enter this
comment and command into your script, and run it:**

```
# Show results of analysis
bf2bs
```

```
Bayes factor analysis
--------------
[1] medit : 718.3828 ±0.01%
[2] sex : 1.446498 ±0.01%
[3] medit + sex : 1160.597 ±1.38%
[4] medit + sex + medit:sex : 83.20431 ±2.23%
Against denominator:
Intercept only
---
Bayes factor type: BFlinearModel, JZS
```

`[1] medit`

This is the **main effect** of training type,
`medit`

. More specifically, it is a test of the hypothesis
that `medit`

, affects `rt`

.

The Bayes Factor for this hypothesis is around 718, which is strong
evidence that there is a main effect of meditation training. If you’ve
got a particularly good memory, you’ll have noticed this isn’t quite the
same Bayes Factor as we saw for the main effect of meditation in this analysis of the same data. As with many
forms of multi-factor ANOVA (including traditional, p-value, ones) the
evidence terms you get depend somewhat on the formula you specify. Here,
you included `sex`

but not `congru`

as factors; in
the previous analysis, it was the other way around. This can affect the
results you get.

If the type of analysis you choose to conduct affects the central conclusions of your study, then that is normally an indication that you need to replicate your result with a larger sample. Conclusions should be robust to variations in analysis technique.

`[2] sex`

This is the **main effect** of participant sex. More
specifically, it is the hypothesis that `sex`

affects
`rt`

. This is compared against the hypothesis that there is
no effect of sex.

The Bayes Factor is about 1.4, meaning the evidence for a main effect of sex is inconclusive.

As before, `anovaBF`

does not directly give us a Bayes
Factor for the interaction of the two main effects. Instead, it gives us
two Bayes Factors for things that we can use to work out the interaction
BF. These are:

`medit + sex`

This the hypothesis that there is a main effect of both factors.
There is no assumption that the two main effects are of the same size.
This ‘main effects’ hypothesis is compared against the null hypothesis,
i.e. that neither `medit`

nor `sex`

affect
`rt`

.

The BF for this hypothesis is large (about 1200). We’d expect this,
given that there was substantial evidence for `medit`

alone.

`medit + sex + medit:sex`

This is the Bayes Factor for the hypothesis that there are main
effects for both factors (`medit + sex`

) *and* that
the two factors interact (`+ medit:sex`

). This is again
compared against the null hypothesis that neither `medit`

nor
`sex`

have any effect.

The BF for this hypothesis is also large (about 85). We’d expect
this, because there was substantial evidence for the ‘main effects’
hypothesis `medit + congru`

. However, the BF is considerably
lower than for the `main effects`

hypothesis.

The Bayes Factor for the interaction is given, as before, by
**entering this comment and command into your script, and running
it:**

```
# Calculate Bayes Factor for interaction
bf2bs[4] / bf2bs[3]
```

```
Bayes factor analysis
--------------
[1] medit + sex + medit:sex : 0.07169096 ±2.63%
Against denominator:
rt ~ medit + sex
---
Bayes factor type: BFlinearModel, JZS
```

This gives us a Bayes Factor for the interaction around 0.07. So,
there’s substantial evidence for the *absence* of an
interaction.

**NOTE:** In this section, we present the simplest way
to perform this calculation. There are better ways. For further
discussion, see Bayesian ANOVA for Pros

We can also use `anovaBF`

to look at two within-subjects
factors. Again, this works just the same way as before. For example, we
can do a factorial Bayesian ANOVA for `congru`

and
`block`

like this:

First, we create the appropriate subject-level summary for our analysis.

**Enter this comment and commands into your script, and run
them:**

```
# Create subject-level summary
wordsum2 <- words %>%
group_by(subj, congru,block) %>%
summarise(rt = mean(rt))
```

Next, for the purposes of this worksheet, we’re also going to reduce the sample size from 420 participants to 60 participants. This is something you’d never do in a real analysis, but here we need to address the issue that within-subject Bayesian ANOVAs can take a very long time to run when we have, as in this case, several hundred participants (in this case, it would take around 15 minutes). The correct approach when analysing your own data is, of course, to just wait for the analysis to finish.

**Enter this comment and command into your script, and run
it:**

```
# Reduce sample size (FOR DEMONSTRATION PURPOSES ONLY !!)
wordsum2 <- wordsum2 %>% filter(as.numeric(subj) < 61)
```

Now we run the Bayesian analysis as before.

**Enter this comment and command into your script, and run
them:**

```
# Calculate Bayes Factors
bf2ws <- anovaBF(
formula = rt ~ congru*block + subj,
data = data.frame(wordsum2),
whichRandom = 'subj')
```

The output is interpreted the same way as before; **enter this
comment command into your script, and run it:**

```
# Display results
bf2ws
```

```
Bayes factor analysis
--------------
[1] congru + subj : 3830.96 ±0.76%
[2] block + subj : 0.03441736 ±0.81%
[3] congru + block + subj : 136.0741 ±1.25%
[4] congru + block + congru:block + subj : 1.543383 ±2.04%
Against denominator:
rt ~ subj
---
Bayes factor type: BFlinearModel, JZS
```

`[1] congru`

The is the **main effect** of congruence. More
specifically, it is the hypothesis that `congru`

affects
`rt`

. This is compared against the hypothesis that there is
no effect of congruence.

The Bayes Factor is very large (about 3,800), meaning the evidence for a main effect of congruence is conclusive.

`[2] block`

This is the **main effect** of test `block`

.
More specifically, it is a test of the hypothesis that
`block`

, affects `rt`

.

The Bayes Factor for this hypothesis is around 0.04, which is strong
evidence for the *absence* of an effect of block.

As before, `anovaBF`

does not directly give us a Bayes
Factor for the interaction of the two main effects. Instead, it gives us
two Bayes Factors for things that we can use to work out the interaction
BF. These are:

`congru + block`

This the hypothesis that there is a main effect of both factors.
There is no assumption that the two main effects are of the same size.
This ‘main effects’ hypothesis is compared against the null hypothesis,
i.e. that neither `congru`

nor `block`

affect
`rt`

.

The BF for this hypothesis is large - about 135. We’d expect this,
given that there was substantial evidence for `congru`

alone.

`congru + block + congru:block`

This is the Bayes Factor for the hypothesis that there are main
effects for both factors (`congru + block`

) *and* that
the two factors interact (`+ congru:block`

). This is again
compared against the null hypothesis that neither `congru`

nor `block`

have any effect.

The BF for this hypothesis is considerably lower than for the
`main effects`

hypothesis.

The Bayes Factor for the interaction is given, as before.
**Enter this command into your script, and run it:**

```
# Calculate interaction Bayes Factor
bf2ws[4] / bf2ws[3]
```

```
Bayes factor analysis
--------------
[1] congru + block + congru:block + subj : 0.01134222 ±2.39%
Against denominator:
rt ~ congru + block + subj
---
Bayes factor type: BFlinearModel, JZS
```

This gives us a Bayes Factor for the interaction is around 0.01. So,
there’s substantial evidence for the *absence* of an
interaction.

It is also possible to use `anovaBF`

to analyse a single
factor, between-subjects design. Where that factor has two levels, you
could alternatively use ttestBF. One
of the useful things about `anovaBF`

is that, unlike
`ttestBF`

, you can use it in a one-factor design that has
more than two levels. For example: (**enter these comments and
commands into your script, and run them:**)

```
# Create subject-level summary
wordsum3 <- words %>%
group_by(subj, medit) %>%
summarise(rt = mean(rt))
# Calculate Bayes Factor
bf1bs <- anovaBF(
formula = rt ~ medit,
data = data.frame(wordsum3))
# Display Bayes Factor
bf1bs
```

```
Bayes factor analysis
--------------
[1] medit : 718.3828 ±0.01%
Against denominator:
Intercept only
---
Bayes factor type: BFlinearModel, JZS
```

Although in principle it is possible to run Bayesian ANOVAs with more
than two factors, they can take hours or days to run with realistically
large data sets. They also produce output that is considerably harder to
interpret than the output of two-factor analyses. For these reasons, we
only look at up to two-factor Bayesian ANOVA in these worksheets. If you
need to conduct an ANOVA with more than two factors, consider using traditional ANOVA techniques (bearing in
mind, of course, the interpretative difficulties surrounding p values -
and that *any* kind of analysis with more than two factors is
often going to be pretty difficult to interpret).

The data for this first exercise comes from a between-subjects
experiment with two factors. Start by loading the data and removing any
missing data (`NA`

), using the following command.

**Enter this comment and commands into your script, and run
them:**

```
# EXERCISE
# Load data
cf_raw <- read_csv('going-further/counterfactual.csv') %>% drop_na()
```

The first few rows of data look like this:

subj | perspective | predictability | blame |
---|---|---|---|

136 | first | Yes | 6 |

137 | first | No | 6 |

138 | first | Yes | 4 |

139 | first | Yes | 6 |

140 | first | Yes | 4 |

141 | first | Yes | 4 |

In this experiment, participants read a scenario describing some events leading up to an accident. The scenario was written from either a first-person or a third-person perspective. Next, they generated some counterfactual thoughts in relation to the scenario. For example, “If only I hadn’t been looking at my phone, then …”. These thoughts were coded as either predictive, or not predictive of whether the accident occurred. Participants rated how much the protagonist was to blame for the accident, on a scale of 1 (strongly disagree) to 7 (strongly agree).

It’s crucial to look at your data before analysing it. Start by
calculating the mean `blame`

score in each of the four
conditions of this experiment. You can do this using the
`group_by`

and `summarise`

commands that we have
covered in many previous worksheets; for example, in the group differences worksheet.

If you’ve done it correctly, your means should look like this:

```
# A tibble: 4 × 3
# Groups: perspective [2]
perspective predictability blame
<fct> <fct> <dbl>
1 first No 4.89
2 first Yes 4.70
3 third No 4.06
4 third Yes 4.38
```

Next, plot those means in a line graph - this helps you visualize the patterns in these data. Plotting of line graphs was covered in the understanding interactions worksheet. Your graph should look like this:

Now, do your Bayesian ANOVA. Set each of the independent variables as
a `factor`

and then use `anovaBF`

to conduct the
Bayesian ANOVA. There is only one observation for each participant in
this dataset, so don’t include `subj`

or
`whichRandom=subj`

.

Your results should look similar to this:

```
Bayes factor analysis
--------------
[1] perspective : 3.252175 ±0.01%
[2] predictability : 0.2035932 ±0.03%
[3] perspective + predictability : 0.6617437 ±2.26%
[4] perspective + predictability + perspective:predictability : 0.3010063 ±4.16%
Against denominator:
Intercept only
---
Bayes factor type: BFlinearModel, JZS
```

```
Bayes factor analysis
--------------
[1] perspective + predictability + perspective:predictability : 0.4548684 ±4.74%
Against denominator:
blame ~ perspective + predictability
---
Bayes factor type: BFlinearModel, JZS
```

**Write a short paragraph explaining what, on the basis of your
analysis, you can conclude from these data. Copy this paragraph, and the
code you used to do your analysis, into PsycEL.**

The data for this second exercise come from a within-subjects
experiment with two factors. Start by loading the data, selecting the
columns and rows we’ll need for this analysis, and removing missing data
(`NA`

).

**Enter these comments and commands into your script, and run
them:**

```
# EXERCISE 2
# Load data, select Test phase and appropriate columns; drop NA entries
raw <- read_csv('case-studies/chris-mitchell/priming-faces.csv') %>%
filter(Running == "Test") %>%
select(Subject, Congruency, Load, RT) %>%
drop_na()
```

The first few rows of data look like this:

Subject | Congruency | Load | RT |
---|---|---|---|

1 | Congruent | NoLoad | 684 |

1 | Congruent | Load | 371 |

1 | Incongruent | NoLoad | 907 |

1 | Incongruent | Load | 659 |

1 | Congruent | NoLoad | 453 |

1 | Congruent | Load | 969 |

Participants were trained to associate two screen colours with pictures of two different food rewards. At test, they saw pairs of screen colours and food pictures. Their reaction time (RT) in milliseconds was measured in cases where the pairs matched the associations they’d previously learned (congruent trials), and in cases where the pairs did not match what they had previously learned (incongruent trials). The participants experienced this test both with, and without, a secondary task. In the secondary task, the participant had to verbally rate how much they liked pictures of faces. As all participants completed all conditions, we have a fully within-subjects design, with factors congruency (congruent, incongruent) and load (load, no load).

Start by calculating the mean RT for each of the four conditions, and plotting these in a line graph. You can use the commands from Exercise 1, slightly modified, to do this. If you get it right, your output should look like this:

```
# A tibble: 4 × 3
# Groups: Congruency [2]
Congruency Load RT
<chr> <chr> <dbl>
1 Congruent Load 672.
2 Congruent NoLoad 575.
3 Incongruent Load 665.
4 Incongruent NoLoad 616.
```

Next, calculate Bayes Factors for the main effects and the interaction. In order to do this, you will first have to create a subject-level summary of your data - you can then pass this your your Bayesian ANOVA.

Your results should look similar to this:

```
Bayes factor analysis
--------------
[1] Congruency + Subject : 0.3059866 ±1.85%
[2] Load + Subject : 19235.62 ±1.11%
[3] Congruency + Load + Subject : 6457.079 ±1.88%
[4] Congruency + Load + Congruency:Load + Subject : 2191.695 ±5.73%
Against denominator:
RT ~ Subject
---
Bayes factor type: BFlinearModel, JZS
```

```
Bayes factor analysis
--------------
[1] Congruency + Load + Congruency:Load + Subject : 0.3394252 ±6.04%
Against denominator:
RT ~ Congruency + Load + Subject
---
Bayes factor type: BFlinearModel, JZS
```

**Write a short paragraph explaining what, on the basis of your
analysis, you can conclude from these data. Copy this paragraph, and the
code you used to do your analysis, into PsycEL.**

This material is distributed under a Creative Commons licence. CC-BY-SA 4.0.