Effect of the Replication Crisis on the Iowa Gambling Study

Abstract

Since 2011 there have been a lot of issues raised about the replication crisis in both the general sciences and in psychology with some people disagreeing with the concept altogether, this is the issue of the majority of replications failing to provide the same results as the original study. The aim of this study was to assess whether the results from the original Iowa Gambling Task could be replicated using the hypothesis that the frequency of selection for advantageous and disadvantageous decks will be the same. To do this a sample of 217 undergraduate psychology students took part in a direct replication of the original Iowa Gambling Task using a repeated measures design. It was found that there was a higher mean frequency selection for advantageous decks in comparison to disadvantageous decks therefore the results found supported the original work by Bechara et al (1994) and disputed the hypothesis. The results from the replication also dispute the idea of the replication crisis as they showed a successful replication despite all the supporting research for the theory.

Introduction

In recent years the issue of the replication crisis has arisen across many areas of science but particularly in psychology, it came to light in 2011 when research involving replications of some of the most famous psychological experiments took place to see if the results could be reproduced. Of the original research 97% found to have significant results but only 36% of the replications found significant results, (Open Science Collaboration, 2015). The replication crisis has resulted in most replications expecting to find different results to the original study due to the high number of replications having shown the original study to be a false positive or have type 1 errors. There have been many examples of failed replications in psychology (Ritchie, Wiseman & French, 2012; Stroebe & Strack, 2014). However the reason for these failed attempts can vary, some are the result of the original studies having false positive results, some are the result of the original study having a small sample as this results in it being less representative so some of the results may be due to chance or the results from the original study are not universal so only apply to a specific time period or group of people (Diener & Biwas-Diener, 2017; Resnick, 2018). Despite all this support there are some people who don’t believe that the replication crisis exists for example Lisa Feldman Barrett (2015) said “the failure to replicate is not a cause for alarm; in fact, it is a normal part of how science works.”.

Get Help With Your Essay

If you need assistance with writing your essay, our professional essay writing service is here to help!
Find out more about our Essay Writing Service

Further contributions to the replication crisis are that often failed replications aren’t published as a result of the file-drawer effect (Rosenthal, 1979), this is the idea that only results that are significant are published and those that aren’t are filed away so there may be multiple non-significant findings hidden that other researchers will not know about. Some replications fail as a result of misconduct within the original study and these misconducts may remain hidden for a considerable time as a result of the file-drawer effect. For example, the case of Diederik Stapel (Varfaellie & McGwin, 2011); it was found that he had altered data and made up experiments in his published work which resulted in his work being difficult to replicate. Similarly, research has found that psychology and psychiatry are 5 times more likely to publish positive results in comparison to other sciences (Yong, 2012) therefore showing a likely cause for why failed replications aren’t published. Another reason for the limited number of replications researched by Yong (2012) is that certain papers will not publish direct replications therefore there a few incentives, so people will be less inclined to do research when they know it will not be published.

There are two methods for replication; direct replication which involves the use of the same methods and materials and conceptual replication, this involves using different methods but still looking at the same theory and using the same hypothesis to see if the results remain significant. Both are used to provide support for the original study or highlight if the original results are a false positive (Diener et al., 2017). Direct replications are favoured over conceptual replication as they are said to have “more confirmatory power” (Noone, 2013), this is because they use the same methods as original studies but test them on a new sample so if the results were truly generalisable then the replication should be successful. Yong (2012) said that few direct replications are done instead people favour conceptual replications as they can change the designs to encourage a positive result which will result in the research being more likely to be published.

The following study involved the use of direct replication of the Iowa Gambling Task (IGT) originally conducted by Bechara, Damasio, Damasio and Anderson (1994). The original study was conducted to see the effect of damage to the ventromedial prefrontal cortex on decision making abilities (Bechara et al, 1994), however for the following study only uses one set of healthy participants to see if the results for the control group in the original IGT were significant. The original study found that the sample with brain damage were oblivious to future implications of their decisions so more of them chose the disadvantageous decks whereas the majority of the control chose from the advantageous decks (Bechara et al, 1994). This study expects to not find the same pattern of results as the original IGT; the frequency of selection for advantageous and disadvantageous decks will be the same.

Method

Participants:

The participants were selected through an opportunity volunteer sample of undergraduate first year psychology students, overall 217 participants were used of which 191 were females and 26 were males, 208 of the participants were right handed and 9 were left handed. The mean age of the sample was 19.47 years and the standard deviation was 1.17 (2 decimal places). This study had ethical approval from a local ethics committee.

Design:

A repeated measures design was used in this study as all the participants took part in all the conditions. The independent variable was whether the deck of cards was a reward or fee and had a nominal level of measurement; decks A and B had higher rewards, but greater fees so were disadvantageous in the long run in comparison to decks C and D which were classed as advantageous in the long run. The dependent variable was the frequency that each deck was picked making it ratio data.

Materials:

Participants were shown a series of four decks of cards on an online test, these decks were labelled either ‘A’, ‘B’, ‘C’ or ‘D’, they were also given a consent form (see Appendix A).

Procedure:

Participants took part in an online test that they were told was a gambling game, they were given a loan of $2000 from the bank and told to make a profit, depending on the card they picked they would receive a reward and/or a pay a fee to the bank. They were then instructed to select a card from one of the four decks on screen and would do this for a total of 100 trials (Bechara et al, 1994). After they had completed the trials the participants were given a survey (see Appendix A) asking if they consent to the use of their data. This all took part in a computer laboratory under exam conditions so there were no distractions.

Results

The analysis of the data revealed that the deck with the highest mean frequency selection was deck C closely followed by D (as can be seen in Figure 1), these advantageous decks had higher means than the disadvantageous decks; the standard deviations, standard errors and confidence intervals were also calculated (see Table 1).

Figure 1. The mean selection frequency for each deck type. The error bars represent the 95% confidence intervals.

Discussion

This study aimed to find the same frequency of selection between types of decks unlike the results found in the study by Bechara et al. (1994), the results demonstrated that the sample chose the advantageous decks in more of the trials than disadvantageous as the means were higher. These results therefore support the original study showing that those results were significant and dispute the hypothesis of this study. The confidence intervals show where 95% of the samples of data lie around the mean, in addition we can be confident that the mean of the population will lie within these limits. Only the confidence intervals for decks ‘C’ and ‘D’ overlap but as they are both advantageous decks this overlap is irrelevant, however the confidence intervals for the advantageous and disadvantageous decks don’t overlap therefore we can be confident that these differences would occur in the population. All the intervals are narrow therefore demonstrating there is only a small variation within results from the mean.

The results from this study dispute the concept of the replication crisis as they supported the original findings from Bechara et al. (1994) due to it being a successful replication. The original IGT was conducted to identify individuals with damage to the ventromedial prefrontal cortex (Bechara et al. 1994) and has since been used as a base for further research into the causes and implications the damage has; the successful replication therefore supports the use of this task within the understanding of the damage. However, this doesn’t completely rule out the replication crisis as many other replication studies have been unable to replicate original findings therefore showing the crisis does exist, for example Ritchie et al. (2012).

The sample used for this study may have an influence over the results as there was a large gender imbalance with there being 191 females and only 26 males. In a recent study sex differences and handedness were researched in relation to the IGT it was found that “right-handedness was associated with advantageous decision-making in the punishment IGT variant, and sex and task motivation appeared to be critical to the right-handed advantage in the punishment IGT variant” (Singh, 2016). This replication mainly involved female participants, so the differences found may be more significant if more males were involved as Singh (2016) said that “male performance is observed to be superior to that of females” as they chose from the advantageous decks at a higher frequency. Similar findings have been demonstrated in research by Van Den Bos, Homberg and Visser (2013); however, research into the effect of testosterone levels on the IGT found that individuals with high levels of testosterone were associated with higher risk taking and so therefore choose from the disadvantageous decks more frequently (Stanton, Liening & Schultheiss, 2011). By having a gender bias in this replication study it may result in unreliable results as if there was a more even ratio then it may be expected to find a higher frequency of choices from the disadvantageous decks.

Another limitation of the conducted study is the age of the participants, in this study the age range was from 18 to 28, with the majority being either 18 or 19. Research into the effect of age difference on decision making in the IGT found that adults avoided disadvantageous decks more than pre-adolescents and adolescents (Cauffman et al, 2010), similarly Mata, Josef, Samanez and Hertwig (2011) said “risk taking peaks in young adulthood”. Therefore, demonstrating that the age of the participants in the study could affect the results however according to the research by Cauffman et al. (2010) the results would show a higher selection frequency of disadvantageous decks but overall the results found would still replicate those found by Bechara et al. (1994).

Research into replicability has resulted in a widespread effort to increase replicability within psychology, for example the Association for Psychological Science (APS) said they have made “changes that encourage greater transparency and stronger statistical analyses and that provide special recognition for pre­registering hypotheses and for sharing materials and data” (Association for Psychological Science, 2015). Similarly, specific places for replication studies to be published have been created for example a website called PsychFileDrawer was created for researchers to put up unpublished replications (Yong, 2012) and Registered Replication Report in Perspectives on Psychological Science (Simons, 2014). Further examples for recommendations on how to reduce the replication crisis include: increasing sample size as this will result in narrower confidence intervals so replicability will be higher and the use of large samples in original studies as this will reduce variance therefore reducing sampling error (Asendorpf et al., 2013). In psychology more replications are required to ensure all research is significant.

References

• Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J., Fiedler, K., … & Perugini, M. (2013). Recommendations for increasing replicability in psychology. European Journal of Personality, 27(2), 108-119.
• Association for Psychological Science. (2015). Report Demonstrates Need for Improved Reproducibility in Psychological Science (pp. 1-2). Retrieved from https://www.psychologicalscience.org/publications/observer/obsonline/report-demonstrates-need-for-improved-reproducibility-in-psychological-science.html
• Bechara, A., Damasio, A. R., Damasio, H., & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition, 50(1-3), 7-15.
• Cauffman, E., Shulman, E. P., Steinberg, L., Claus, E., Banich, M. T., Graham, S., & Woolard, J. (2010). Age differences in affective decision making as indexed by performance on the Iowa Gambling Task. Developmental psychology, 46(1), 193.
• Diener, E., & Biswas-Diener, R. (2017). The replication crisis in psychology. Noba Textbook Series: Psychology, eds Biswas-Diener R, Diener E (DEF Publishers, Champaign, IL).
• Feldman Barrett, L. (2015). Opinion | Psychology Is Not in Crisis. Retrieved from https://www.nytimes.com/2015/09/01/opinion/psychology-is-not-in-crisis.html
• Mata, R., Josef, A. K., Samanez‐Larkin, G. R., & Hertwig, R. (2011). Age differences in risky choice: A meta‐analysis. Annals of the New York Academy of Sciences, 1235(1), 18-29.
• Noone, C. (2013). REPLICATION STUDIES: IT’S TIME TO CLEAN UP YOUR ACT, PSYCHOLOGISTS! [Blog]. Retrieved from https://blog.efpsa.org/2013/01/30/replication-studies-its-time-to-clean-up-your-act-psychologists/
• Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.
• Resnick, B. (2018). More social science studies just failed to replicate. Here’s why this is good. Retrieved from https://www.vox.com/science-and-health/2018/8/27/17761466/psychology-replication-crisis-nature-social-science
• Ritchie, S. J., Wiseman, R., & French, C. C. (2012). Failing the future: Three unsuccessful attempts to replicate Bem’s ‘Retroactive Facilitation of Recall’Effect. PloS one, 7(3), e33423.
• Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological bulletin, 86(3), 638.
• Simons, D. J. (2014). The value of direct replication. Perspectives on Psychological Science, 9(1), 76-80.
• Singh, V. (2016). Sex-differences, handedness, and lateralization in the Iowa gambling task. Frontiers in psychology, 7, 708.
• Stanton, S. J., Liening, S. H., & Schultheiss, O. C. (2011). Testosterone is positively associated with risk taking in the Iowa Gambling Task. Hormones and behavior, 59(2), 252-256.
• Stroebe, W., & Strack, F. (2014). The alleged crisis and the illusion of exact replication. Perspectives on Psychological Science, 9(1), 59-71.
• Van den Bos, R., Homberg, J., & de Visser, L. (2013). A critical review of sex differences in decision-making tasks: focus on the Iowa Gambling Task. Behavioural brain research, 238, 95-108.
• Varfaellie, M., & McGwin, J. (2011). The case of Diederik Stapel. Retrieved from https://www.apa.org/science/about/psa/2011/12/diederik-stapel.aspx
• Yong, E. (2012). feature; In The Wake Of High-profile Controversies, Psychologists Are Facing Up To Problems With Replication. Nature, 485(7398), 298-300.

Appendices

Appendix A

Practical 1: The Replication Crisis

Have you just completed the practical task? If so, great. If you are happy for your data to contribute to the practical, then you can complete the form below.

Required

1. I understand that my participation is voluntary and that I am free to withdraw at any time without giving any reason and without there being any negative consequences. In addition, should I not wish to answer any particular question or questions, I am free to decline.

Yes

No

2. I give permission for members of the research team to have access to my anonymised responses. I understand that my name will not be linked with the research materials, and I will not be identified or identifiable in the report or reports that result from the research. If information collected in this study is published in scientific journals, where necessary, participants will be referred to by an anonymous code only. The terms of the data protection Act 1988 will be adhered to and information will be securely stored.

Yes

No

3. I agree for the data collected from me to be used in future research projects and teaching materials.

Yes

No

4. Student ID

5. Date of Birth

6. Gender

Male

Female

Prefer not to say

7. Handedness (i.e. which hand was your mouse in to complete the task?)

Right

Left

8. Make sure to paste in all of the data from the webpage here.

Submit