## Before you start

This is an advanced worksheet, which assumes you have completed the Absolute Beginners’ Guide to R course, the Research Methods in Practice (Quantitative section) course, and the Intermediate Guide to R course.

## Introduction

This worksheet describes a partial analysis pipeline for an experiment taken from a PhD thesis. The experiment explored self-esteem before and after a mental imagery intervention. Self-esteem was measured using the State Self-Esteem Scale (SSES, Osman et al., 2012), a 20-item survey used to measure short-lived (state) changes in self-esteem. Participants completed one of two mental imagery conditions, or a control condition:

• Condition 1: participants visualised a negative mental image of themself
• Condition 2: participants visualised a negative mental image of someone else
• Condition 0 (control): participants did a card sorting task and did not think of any images.

This was a 2 (time) x 3 (condition) mixed design.

## Preprocessing

The initial preprocessing steps for this data are described in the Data preprocessing for scales worksheet. Complete that worksheet first, then add the code in this worksheet to the end of scales.R and run it.

We start with some final bits of preprocessing. Enter these commands into your script, and run them:

sses_raw <- bind_rows(sses_pre_raw, sses_post_raw)
sses_raw <- sses_raw %>%
mutate(total = sses_raw %>% select(q1:q20) %>% rowSums())

### Explanation of commands

bind_rows() simply joins data frames together, in the order the arguments are specificed, making a new data frame. We do this because we want to do at least one analysis on all of the data.

The mutate() in the second line uses rowSums() to add up the values in the columns q1:q20 for each row. Because our data has one participant per row, this calculates the SSES score for each participant. You’ve seen something similar when we calculated subscale scores in the Data preprocessing for scales. The SSES contains some reverse-coded items, but these were already reversed in this particular dataset, so we didn’t need to do that step.

## Calculating descriptive statistics

We can get a feel for our data by using the familiar group_by() and summarise() commands to calculate means and standard deviations by group within time.

Enter these commands into your script, and run them:

sses_raw %>%
group_by(time, condition) %>%
summarise(mean = mean(total), sd = sd(total))
summarise() has grouped output by 'time'. You can override using the .groups argument.
# A tibble: 6 x 4
# Groups:   time [2]
time  condition  mean    sd
<fct> <fct>     <dbl> <dbl>
1 pre   control    31.1  9.34
2 pre   other      27.8  6.79
3 pre   self       33.8  9.27
4 post  control    29.3  9.19
5 post  other      28.6  7.44
6 post  self       37.7  7.58

From these results, we can see that, compared to the pre-intervention scores, self-esteem rose slightly in other condition, rose even more in the self condition and dropped slightly in the control condition.

## Checking SSES reliability

To check the reliability of our SSES measurements, we’ll calculate Cronbach’s alpha for the pre-intervention SSES data. Cronbach’s alpha was introduced in the Analysing scales worksheet.

Enter these commands into your script, and run them:

library(psy)
sses_pre <- sses_raw %>% filter(time == 'pre')
sses_pre %>% select(q1:q20) %>% cronbach()
$sample.size [1] 73$number.of.items
[1] 20

$alpha [1] 0.6862132 ### Explanation of commands Line 1 loads the psy package which provides the cronbach() function. Line 2 creates a data frame contain only the rows with pre-intervention scores, along with the condition, and SSES columns. We only use the pre-intervention scores because, assuming the interventions were successful, including the post-intervention scores would reduce alpha. The last line calcluates Cronbach’s alpha. ### Explanation of output The value of alpha (0.69) could be a cause for concern, as it is below the 0.7-0.8 convention of acceptable reliability, and well below the alpha of 0.92 reported by the authors of the SSES (Heatherton & Polivy, 1991). We’ll do an additional analysis to see how the scale performed in each group. Enter these commands into your script, and run them: sses_pre %>% filter(condition == 'control') %>% select(q1:q20) %>% cronbach() $sample.size
[1] 34

$number.of.items [1] 20$alpha
[1] 0.7419521
sses_pre %>% filter(condition == 'self') %>%
select(q1:q20) %>% cronbach()
$sample.size [1] 18$number.of.items
[1] 20

$alpha [1] 0.6823747 sses_pre %>% filter(condition == 'other') %>% select(q1:q20) %>% cronbach() $sample.size
[1] 21

$number.of.items [1] 20$alpha
[1] 0.4775607

### Explanation of commands

Lines 1-2 filter the pre-intervention data to only include the control condition, and then calculates Cronbach’s alpha. The remaining lines do the same for the other two conditions.

### Explanation of output

We can now see from these results that alpha was especially low in the self and other conditions.

## Comparing baseline SSES scores

Putting aside the reliability of the measurements for now, we would like to check that there were no major self-esteem differences between conditions before our intervention. This should be the case if we successfully randomised participants to conditions. If there are baseline differences we would need to account for these when comparing them to post-intervention scores. We can use a between-subjects ANOVA to compare the baseline SSES scores in our three conditions. This is the similar to the ANOVA described in the Within-subject differences worksheet, but for a between-subjects factor.

Enter these commands into your script, and run them:

library(BayesFactor, quietly = TRUE)
sses_pre <- sses_raw %>% filter(time == 'pre')
anovaBF(formula = total ~ condition, data = data.frame(sses_pre))
Bayes factor analysis
--------------
[1] condition : 0.6820205 ±0.03%

Against denominator:
Intercept only
---
Bayes factor type: BFlinearModel, JZS

### Explanation of commands

In line 1, we load the BayesFactor package. Line 2 assigns just the pre-intervention data to sses_pre. Line 3 runs the between-subjects ANOVA. To run a Bayesian ANOVA using a random factor, we would need more than one observation for each participant for condition. As we only have one observation for each participant, we don’t use + subj in the formula, and whichRandom = 'subj'.

### Explanation of output

The Bayes factor of .68 is greater than the conventional 0.33 which would satisfy us that there were no differences between the conditions. This could be due to the slightly lower mean in the other condition. We’ll address this issue at the end of the worksheet.

Enter these commands into your script, and run them:

sses_raw %>% filter(time == 'pre') %>%
group_by(condition) %>%
summarise(mean = mean(total), sd = sd(total))
# A tibble: 3 x 3
condition  mean    sd
<fct>     <dbl> <dbl>
1 control    31.1  9.34
2 other      27.8  6.79
3 self       33.8  9.27

## Comparing interventions

The main question we’d like to answer is whether our two imagery interventions (visualising a negative mental image of oneself, or someone else) had effects on self esteem which differed from our control condition. We can test this using a factorial ANOVA to compare SSES scores before and after the three interventions. In this design, condition is a between-subjects variable, and time (pre and post intervention) is a repeated measure.

Enter these commands into your script, and run them:

bf <- anovaBF(formula = total ~ time*condition + subj,
data = data.frame(sses_raw), whichRandom = 'subj')
bf
Bayes factor analysis
--------------
[1] condition + subj                         : 3.735972  ±2.28%
[2] time + subj                              : 0.1908497 ±1.19%
[3] condition + time + subj                  : 0.7704205 ±2.88%
[4] condition + time + condition:time + subj : 7.014156  ±4.47%

Against denominator:
total ~ subj
---
Bayes factor type: BFlinearModel, JZS
bf[4] / bf[3]
Bayes factor analysis
--------------
[1] condition + time + condition:time + subj : 9.104323 ±5.32%

Against denominator:
total ~ condition + time + subj
---
Bayes factor type: BFlinearModel, JZS

### Explanation of commands

The first two lines run a Bayesian factorial ANOVA, with subj as a random factor, and store result is stored in bf. Line 3 prints the results, which provides us with the Bayes Factors for the main effects of condition (bf[1]) and time (bf[2]). Line 4 calculates the Bayes Factor for the interaction.

### Explanation of output

The Bayes Factor for time is less than .33, which tells us that there was no overall change in self-esteem after the intervention relative to baseline. The Bayes Factor for condition is greater than 3, indicating that there were differences in self-esteem between the three conditions. The final Bayes Factor tells us that it’s about 11 times more like that there’s an interaction between condition and time than that there isn’t.

# References

Heatherton, T. F., & Polivy, J. (1991). Development and validation of a scale for measuring state self-esteem Journal of Personality and Social Psychology, 60(6), 895.