Rescorla Wagner Learning

Rescorla Wagner Learning

Critically evaluate the contributions of the Rescorla-Wagner model to our understanding of associative learning.

The Rescorla-Wagner model is widely regarded as the most influential and groundbreaking theory of associative learning, providing a clear mathematical solution to the complex phenomena of classical conditioning.

Early theories on classical conditioning for many years subscribed themselves to Pavlov’s (1927, as cited in Gross 1994) simple explanation to of association by contiguity, whereby continuous presentation of a conditioned stimulus (CS), repeatedly paired with an unconditioned stimulus (US), causes strengthening of the associations until eventually the CS will elicit the same response as the US despite its absence (Lieberman 2000).

However, several problems arise when explaining classical conditioning through contiguity and the theory is unable to account for several specific effects seen in classical conditioning experiments such as contingency and blocking. For example Rescorla (1966) demonstrated that simply pairing a CS and US does not always produce a conditioned response to the CS, showing that the occurrence of the US in absence of the CS is also a critical factor that influences conditioning dramatically. This was demonstrated using a Sidman avoidance task with dogs.

Here three conditions existed. The US (in this case a shock) was either paired or explicitly unpaired with the CS (a light), or a random presentation of the CS and the shock was given. The conditioned response to the US was avoidance behaviour displayed by the dogs, in this case, moving in order to avoid the shock. It was shown that dogs in the paired condition showed more avoidance behaviour than the dogs in the random presentation condition despite the fact that CS-US pairings were equal in number for both conditions. This demonstrates that more is being learned than just the pairing of CS and US.

The Rescorla-Wagner model is able to explain contingency using a single mathematical equation:

∆V_X = αβ(λ–V_sum).

Where ∆V_X is the change in associative strength of stimulus X as a result of the current trial, V_sum is the combined associative strength of all CS present in the trial. λ is the maximum associative strength that a US can support. While α and β represent parameter constants of the CS and US respectively, based on their particular properties.

The model is rooted in the concept of surprise which was first suggested by Kamin (1969) through the discovery of the blocking effect. Kamin observed that if a CS compound (e.g. a tone and a light) were followed by a US (a shock) no fear would be conditioned to the light if the tone had previously been conditioned to predict the US. Therefore the early association formed between the light and shock will block subsequent learning that a tone also predicts the shock. It was suggested that this failure of learning is due to the surprise of the US. If the shock is surprising to the subject, then a search for an explanation occurs. If however an explanation is already present, no searching will occur (Lieberman 2000).

The Rescorla-Wagner model adds further to this, stating that level of expectancy accounts for the degree of conditioning occurring on each trial. For example, an unexpected US will result in significantly more conditioning than if the US was more expected. On a trial by trial basis, the associative strength of a CS will grow towards a stable level (the asymptote) but at a decelerated rate due to the CS eliciting less surprise and therefore greater expectation of a US as more training occurs.

This type of acquisition curve is commonly seen in classical conditioning experiments and the model does in fact account for this occurrence by predicting that the difference between the asymptote level (λ) and current level of association between the CS and US deceases across trails (Miller, Barnet & Nicholas 1995).

As λ represents the level strength of association that would be achieved after continuous CS-US presentations, it follows that if the CS is presented in absence of the US, this value should be set to 0, since no association can be formed. By adopting this value, the Rescorla-Wagner model is capable of explaining the phenomena of extinction. Extinction occurs when the CS is no longer followed by a US and the previously conditioned response diminishes and finally becomes extinct.

Since the associative strength in an extinction experiment is initially higher than λ (0) due to previous acquisition trials, the Rescorla-Wagner model will calculate the change in the strength of CS-US association (ΔV) as a negative value, therefore the strength will decrease by this value in the given trial and in turn will lower the value for V in the following trial. This would explain Rescorla’s (1966) early results with conditioning in dogs. In trails which had presented a light with no shock, the strength of association was being decreased by ΔV, whereas dogs who never experienced a shock in the absence of a light did not loose any associative strength.

The Rescorla-Wagner model can successfully predict and explain both acquisition and extinction seen in classical conditioning, and therefore can be regarded as of great benefit to our understanding of basic associative learning.

A further effect the model can predict is blocking. It does this by taking into account the associative strength of each CS presented and how they contribute to overall strength (V_sum). The model assumes that this total associative strength is equal to the sum of the strength of each CS. Therefore in blocking experiments such as Kamin (1969), it is able to explain the lack of learning occurring for the added stimulus in a compound trial.

For example, the associative strength of the tone in initial training had effectively reached its maximum level, λ. Therefore the discrepancy between V_tone and λ would be near 0. Adding the associative strength of the second stimulus (light) to the first, results in the same value since no associative strength would exist for the light as no training had occurred. Therefore, this combined value for V_sum is still effectively equal to λ. The difference between these two values shows the amount of conditioning and in this case it will be 0, predicting precisely what is seen in blocking experiments, no further conditioning.

The models assumption that the associative strength of a compound is equal to the sum of all CS present can however cause some difficulties in its predictions. As V_sum is greater than either CS associative strength alone, the model will predict that presentation of both CS at once will elicit a greater response and therefore greater conditioning than if each CS was presented in isolation. However, research has shown than this prediction can be disproved. Bellingham, Gillette-Bellingham and Kehoe (1985, as cited in Lieberman, 2000), in an experiment using rats paired a US of water with a light and a tone individually, so each separately predicted the water.

This was integrated with trials where both stimuli were presented in compound, followed by no water. Interestingly the rats showed significantly less responding when the compound was presented, but retained conditioning to the separate stimuli. This demonstrates a key failure of the Rescorla-Wagner model and its inability to predict new experimental observations of classical conditioning since its development. It would appear that the rats were learning that the compound was a separate stimulus in itself which did not predict the water. This observed phenomena can however been explained through the configural learning approach (Peace, 1994 as cited in Redhead, 2007).

This theory is different from elemental approaches (such as the Rescorla-Wagner model) in that it suggests a configural representation of the compound is formed (in this case the light and tone), which subsequently leads to learning of its unique prediction. Despite treating the compound in this way, configural models are still able to predict responding to a CS presented by itself after compound training by assuming associative strength is built through generalisation to the other stimuli depending on their similarity. Thus, configural models may be a better aid to out understanding of associative learning as they account for a larger range of effects than elemental models such as the Rescorla-Wagner model. This example has highlighted a problem with the Rescorla-Wagner models rule of summing associative strengths.

A second problem that can be seen with the model is demonstrated through spontaneous recovery. This is where a CS elicits a response to the US when presented by itself, despite being previously extinguished during presentations of the CS in absence of the US. This occurrence is seen when a rest period follows extinction trails. The Rescorla-Wagner model views extinction as a complete loss of associative strength, with the predictability of the CS being unlearned (Miller et al., 1995).

It is therefore unable to account for this spontaneous recovery and responding after a rest period since it will predict the CS associative strength as 0 following extinction. Furthermore, retraining the CS-US association after extinction, often results in far more rapid learning than in initial acquisition trials. Again, the model fails to predict this successfully since its rigid formula predicts re-learning in a similar fashion seen in initial acquisition trials. This learning rate is predicted by the change of associative strength of each trial (ΔV).

Evidence from Napier, Macrae, & Kehoe (1992) demonstrates this. Here rabbits were trained to respond to a tone by pairing it with shock stimulation to the nictitating membrane on the eyelid. Following extinction trials, it was found that eyelid movement response was learned at a faster rate by CS-US pairings when it was tested in subsequent CS alone trials compared to the rate of initial learning. It would appear from this, despite full extinction, learning can occur at a faster rate than predicted by the Rescorla-Wagner model.

This is likely to be due to its assumption that past associations do not play a role in its current status of the CS. Memory traces may also play a role here since reintroducing CS-US relationships after only a short time may trigger a recollection and cause associations to strengthen more rapidly (Miller at al., 1995). The Rescorla-Wagner model would need revisions in order to account for these discussed effects and to be of further use to our understanding of associative learning.

The CS pre-exposure effect has also caused some difficulty for the Rescorla-Wagner model and has hindered its usefulness to aid our understanding of associative learning to some degree through its inaccuracy to predict effects observed in experimental work which offer the subject pre-exposure to a CS. One of the to-be conditioned stimuli repeatedly presented in absence of a US can cause a reduction in its ability to gain associative strength when later paired with it. Lubow and Moore (1959) demonstrated this effect using bovid animal subjects.

Here, one sample of animals were pre-exposed to a light stimulus with no US while a control sample had no experience of it. It was found in subsequent training that conditioning occurred significantly slower in the sample previously pre-exposed to the light than in the control sample. However the Rescorla-Wagner model will predict an equal rate of training in both conditions, since no previous associative strength has being built with the US in either. The asymptote level will be predicted as 0 when no US is present in the pre-exposure trials since no association can be formed with it.

Further, the CS associative strength (V) will also be 0 since the light is a novel stimulus. Since ΔV on any given trial is calculated by this difference multiplied by the CS and US effect constants, this value will predict no conditioning. The effect seen in the Lubow and Moore experiment can however be explained through attention (Miller et al., 1995). Research shows that repeated presentation of a stimulus can result in habituation and a lack of attending to it. For example (Kaye and Pearce, 1987, as cited in Lieberman, 2000) shows that rats in a stimulus familiar group showed far less orienting towards a light when it was switched on compared to a stimulus novel group.

Thus, it is likely that pre-exposure effects are caused by stimulus habituation and not by the acquisition of response inhibiting characteristics of a CS. The Rescorla-Wagner model cannot predict this effect because it does not take into account the amount of attention to a stimulus (Lieberman, 2000).

Overshadowing is classical conditioning phenomenon which is explained well by the Rescorla-Wagner Model. If two neutral stimuli are paired together in a compound to predict a US in training trials, it can be found that when these stimuli are later tested, one will elicit less conditioned responding than if it had been paired with the US in the absence of the second stimulus (Cole, Oberling & Miller 1999).

For example, if a CS₁ and CS₂ compound is used to predict a US, it can been seen that CS₂ elicits less conditioned response than if it had been paired with the US independently.The model predicts this accurately by assuming that the overshadowing stimulus is taking up some of the maximum associative strength the US is able to sustain, which would not occur in its absence. Therefore less conditioning to the overshadowed stimulus is possible compared to when it is presented exclusively with the US (Miller et al., 1995). This effect was first observed through Pavlov’s (1927, as cited in Gross, 1994) early work on classical conditioning.

Despite the success of the models formula with regards to overshadowing, further research into this area causes it difficultly. The model is not able to explain one-trial overshadowing. This is where one CS overshadows the other on the first training trial. According to the Rescorla-Wagner model this should not happen because both stimuli are initially neutral, occupying none of the associative strength sustained by the US (Miller, 1985). Overshadowing is typically seen when the overshadowing stimulus is more salient than the overshadowed stimulus.

Mackintosh and Reese (1979, as cited in Cole, 1999) have provided an explanation of one-trial overshadowing based on an attentional model of learning. They suggest that in a single compound trial where one stimulus is more salient than the other, the more salient stimulus gains greater perceptual attention over the less salient stimulus. The overshadowed stimulus will not be affected by less perceptual attention when presented alone with the US. Overshadowing is caused by the decrease in perceptual attention which causes a lack of processing of the less salient CS to occur on the same trial.

There appear to be many effects that the Rescorla-Wagner cannot predict based on its failed assumptions; however there are also examples of effects which the model cannot accommodate at all (Miller et al., 1995). One such example of this is the ability animals appear to have in learning that a particular stimulus only predicts an outcome when it preceded by another distinct stimulus. This is known as facilitation. For example, (Rescorla, 1985, as cited in Rescorla 1987) presented pigeons with an auditory stimulus, followed by a light which when pecked delivered a food reward. When trained in such a way, it was found that the pigeons learnt to peck the light predominantly when it was signalled by a tone compared to when presented alone.

The Rescorla-Wagner model can account for this as it predicts associative strength is built in both stimuli as a result of presenting them before the food reward. When both are presented, a greater predictive strength occurs compared to when just one is presented, accounting for an increased response rate. Extinction trials were subsequently carried out to ensure than this observed result was not simply due to the summed associative strength of both stimuli being a better predictor of the food than the associative strength of the light alone. After presenting the noise with no reward across trials, causing complete extinction, the test trial was run again.

It was found that the pigeons still showed increased pecking when the light was preceded by the tone compared to when the light was shown by itself. This indicates that no association was built between the noise and food, and that the associations between them are not important (Lieberman, 2000). Both Holland (1983, as cited in Rescorla, 1987) and Rescorla (1985, as cited in Rescorla, 1987) suggest that results such as these occur because the light becomes associated with the food, while the tone acts as a modulator which facilitates the association between the light and food. Therefore it can be argued that the Rescorla-Wagner model cannot explain this purely through its use of CS-US relationships, and other factors can be present in associative learning contexts.

The Rescorla-Wagner model clearly has some problems in its rigid assumptions which do not allow it to explain the vast number of effects seen across associative learning research. However, it has remained a prominent model and has seen widespread success in many areas of psychological research, explaining and predicting a wide range of phenomena. Its abilities to successfully account for the fundamental principles behind classical conditioning should not be overlooked when examining its many flaws.

References

Cole, R., Oberling, P., & Miller, R. (1999) Recovery from one-trial overshadowing. Psychonomic Bulletin & Review, 6, 424-431.

Gross, R. (1994). Key studies in psychology (2^nd ed.). Norwich: Hodder & Stoughton.

Kamin, L. J. (1969). Predictability, surprise, attention and conditioning. In Campbell, B., and Church, R. Punishment and Aversive Behaviour (pp. 279-296). New York: Appleton-Century-Crofts.

Lieberman, D. (2001). Behavior and Cognition (3^rd ed.) USA: Wadsworth.

Lubow, R., & Moore, A. (1959). Latent inhibition: The effect of nonreinforced preexposure to the conditioned stimulus. Journal of Comparativeand Physiological Psychology, 52, 415-419.

Miller, R., Barnet, R., & Nicholas, G. (1995). Assessment of the Rescorla-Wagner Model. Psychological Bulletin, 117, 363-386.

Napier, R., Macrae, M., & Kehoe, E. (1992). Rapid reacquisition in conditioning of the rabbit's nictitating membrane response. Journal of Experimental Psychology: Animal Behavior Processes, 18, 182–192.

Redhead, E. (2007). Multi-modal discrimination learning in humans: evidence for configural theory. Quarterly Journal of Experimental Psychology, 60, 1477-1495.

Rescorla, R. (1966). Predictability and number of pairings in Pavlovian fear conditioning. Psychonomic Science, 4, 383-384.

Rescorla, R. (1987). Facilitation and Inhibition. Journal of Experimental Psychology: Animal Behavior Processes, 13, 250-259.