Features of Classical Conditioning

<H1 Align="Center">Features of Classical Conditioning</H1> <H2>Factors Determining the Effectiveness of Classical Conditioning</H2> <P>In addition to food induced salivation other reflexes commonly used include access to an opposite-sex conspecific in order to condition courtship behavior in birds, eye- blinks (or nictitating membrane closure in animals like rabbits) elicited by puffs of air directed at the eye, leg-withdrawal from electric shock, together with the more complex constellation of 'fear' reactions to shock such as changes in skin-conductivity (galvanic skin response, GSR), changes in heart-rate and suppression of ongoing behaviors (when these responses are conditioned they are known as a conditioned emotional response - CER), the reflexive pecking which food elicits in birds, or the gagging and nausea which the flavour of a poisonous food comes to elicit. It must be emphasised that the these reflexes may exist in some species but not others. Given an effective US, however, there are sill many factors which influence conditioning a particular CS using it. </P> <H3>Timing</H3> <P>The precise timing of the CS-US pairings has a great influence on whether the CS can be reliably conditioned to produce the CR. To begin with, a closer look at the development of the CR: </P> <P><IMG SRC="devofcr.gif" Alt="development of the CR"></P> <Font Size=-3> <P>This diagramm represents the occurrence of events over time. Each sort of event is represented on a separated line. The points or regions over which this line is raised denote the times when the event is occurring. The diagramm shows how the salivation response of a dog changes as the presentation of food, the US, is repeatedly paired with a tone, the CS.</P> </Font> <P>As can be seen, the dog initially only salivates when presented with the food US, but gradually the tone begins to elicit salivation, until, after a number of trials the dog salivates as soon as the tone is perceived. </P> <P><IMG SRC="cs_us_ti.gif" Alt="CS-US timing"></P> <P>As the figure above shows, the relative timing of the CS and US is crucial to the success of conditioning. Most of these results appear to have simple explanations. In delay conditioning the salivation elicited by the US gradually extends backwards as it becomes associated with the US. In the standard paradigm the same thing happens although it is a little less obvious because of the overlap between the CS and US. In trace conditioning one might assume that the very recent memory trace of the CS begins to be associated with the US and hence the UR gradually extends back, albeit weakly, to the actual occurrence of the CS. The strength of trace conditioning is generally inversely related to the delay between the CS and US. The same cannot be happening in backwards conditioning which is usually ineffective. Once the CS appears, the US has already elicited a response so the effectiveness of conditioning is not simply dependent on the temporal contiguity of the CS and US. Thus a crucial feature of successful CS-US pairing in time seems to be that the CS can be used to predict the occurrence of the US. This suggestion may also explain one of the most surprising results - if the CS and US are presented exactly simultaneously, so that both the onset and offset both occur together, then conditioning fails. </P> <H3>Predictability</H3> <P>The results of different <A HREF="timing.html">timing </A>procedures suggests that an important determinant of the success and strength of conditioning might be the extent to which the CS predicts the US. Is ? In order to examine whether predictability is important above and beyond the number of times the CS and US occur together, let's require a procedure in which the correlation between CS and US occurrence is varied while the total number of times the subjects are exposed CS-US pairings are held fixed. An experiment which examined this question was devised by Rescorla (1968). He took four groups of rats and exposed them all to tone-shock pairings. In each test session the animals heard a number of 2 minute tones interspersed by silences. For all groups the probability that they would experience a shock while hearing the tone was 0.4. The groups differed in the probability that they were also shocked during the silences occurring between tones - one group received no shocks during these intervals (a probability of 0.0), the other groups received shocks during the 'no tone' intervals with probabilities of 0.1, 0.2 and 0.4. So, although all the groups receive the same number of tone-shock pairings the tone becomes a progressively better predictor of shock as the probability of shock occurring during the 'no-tone' intervals decreases. He also tested a control group who received no shocks but heard the same number of tones as the experimental groups. The results clearly show that CS-US predictability is an important factor in determining the efficacy of conditioning - the more the experimental groups were shocked during no-tone intervals the less the tone could predict shock and the less their bar pressing was suppressed by the tone during the <A HREF="suppress.html">conditioned suppression</A> test-phase of the experiment. </P> <P><IMG SRC="rescorla.gif" Alt="prediction"></P> <P>This experiment was the forerunner of many others which were used in attempts to discover precisely how animals use information available to them about stimuli to learn about their environment. Two other phenomena concerned with the way a CS can be used to predict the occurrence of a US form the building blocks for this work - <A HREF="overshadowing.html">overshadowing </A>and <A HREF="blocking.html">blocking</A>. </P> <H3>Overshadowing</H3> <P>In trying to explain the classical conditioning process, experiments where two (or even more) CSs are presented with a single US were conducted. By varying aspects of the CSs and comparing the effectiveness of their conditioning to the single US the scientists hoped to discover what properties of the CS (and, eventually the US), as opposed to the CS-US pair, determine the effectiveness of conditioning. Probably the simplest experiment comparing two CSs is a demonstration of the phenomenon of overshadowing. In this experiment two CSs, CS1 and CS2, are always presented together during training. In the test-phase, the strength of conditioning to the stimuli CS1 and CS2 presented individually are measured. The typical outcome shows that the strength of conditioning to each CS depends on their relative intensity. If CS1 is a dim light and CS2 a bright light then, after conditioning to the CS1-CS2 combination, the CR to the bright light is very strong while the dim light alone produces little or no reaction. The general perceived strength of stimuli is commonly referred to as their <I>salience</I>. Although it might be related to the physically measurable intensity of stimuli, salience is refers to the intensity of the subjective experience of stimuli, not of the objective intensity of the stimuli themselves. Salience, as subjective experience, varies between individuals, and, more importantly, between species. Salience is depends on some combination of the physical characteristics of stimuli and of the sensory systems of the perceiver. </P> <H3>Blocking</H3> <P>In addition to variations in the subjective characteristics, variations in the history of these experiences were also investigated upon . Suppose that two stimuli CS1 and CS2 will be paired with a single US. Rather than presenting the two CSs together throughout the training of an animal only CS1 is used in the first half of training and later the CS1-CS2 combination together, just as in an overshadowing experiment for the second half of training. The result, in general, is that when subsequently tested individually the animal will show strong conditioning to CS1 and little or no conditioning to CS2. The effect where the prior pairing of one stimulus with a US stops the US being associated with other subsequently presented stimuli is called blocking. </P> <P>Procedure for a simple blocking experiment:</P> <table border> <td>Group Name<td>1st Training<td>2nd Training<td>Test<tr> <td>Blocking<td>CS1-US<td>CS1-CS2 and US<td>CS2 alone<tr> <td>Control<td>nothing<td>CS1-CS2 and US<td>CS2 alone<tr> </table> <P>To control for the possible confounding effects of overshadowing, experiments were conducted, in which the roles of CS1 and CS2 were reversed so that CS2 was experienced paired by itself with the US in the 1st training before the 2nd training with the CS1-CS2 compound stimulus. In this experiment normally strong conditioning to CS2 and little conditioning to CS1 would be expected. </P> <P>In order to compare the effectiveness of overshadowing vs. blocking and include some measure for the predictability of the CSs, a "blocking and predictability" experiment could be devised:</P> <table border> <td>Group Name<td>1st Training<td>2nd Training<td>Test<tr> <td>Correlated<td>Correlated CS1 and US<td>CS1-CS2 and US<td>CS2 alone<tr> <td>Uncorrelated<td>Uncorrelated CS1 and US<td>CS1-CS2 and US<td>CS2 alone<tr> <td>Overshadowing<td>US alone<td>CS1-CS2 and US<td>CS2 alone<tr> <td>No US<td>CS1 alone<td>CS1-CS2 and US<td>CS2 alone<tr> </table> <P>Testing the strength of association between CS2 and the US using a <A HREF="suppress.html">suppression ratio procedure</A>, Rescorla (1971) found the results depicted in the diagramm below: the strength of conditioning to CS2 acquired during 2nd training was much weaker in the group which had received prior correlated parings of CS1 and the US than in the groups which had received no prior pairings (overshadowing) or which had received random presentations of CS1 and the US. Finally, the group which had received CS1 alone with no US in phase 1 showed even stronger conditioning to CS2 than the overshadowing or uncorrelated controls. </P> <P><IMG SRC="blocking.gif" Alt="blocking"></P> <P></P> <H2>Rescorla and Wagner's model of classical conditioning.</H2> <P>According to Rescorla and Kamin, associations are only learned when a surprising event accompanies a CS. In a normal simple conditioning experiment the US is surprising the first few times it is experienced so it is associated with salient stimuli which immediately precede it. In a blocking experiment once the association between the CS (CS1) presented in the first phase of the procedure and the US has been made the US is no longer surprising (since it is predicted by CS1). In the second phase, where both CS1 and CS2 are experienced, as the US is no longer surprising it does not induce any further learning and so no association is made between the US and CS2. This explanation was presented by Rescorla and Wagner (1972) as a formal model of conditioning which expresses the capacity a CS has to become associated with a US at any given time. This <I>associative strength</I> of the US to the CS is referred to by the letter V and the change in this strength which occurs on each trial of conditioning is called dV. The more a CS is associated with a US the less additional association the US can induce. This informal explanation of the role of US surprise and of CS (and US) salience in the process of conditioning can be stated as follows: </P> <P><B>dV = ab(L - V)</B></P> <P>where a is the salience of the US, b is the salience of the CS and L is the amount of processing given to a completely unpredicted US. In words: when the US is first encountered the CS has no association to it so V is zero. On the first trial the CS gains a strength of abL in its association with the US which is proportional to the saliences of the CS and the US and to the initial amount of processing given to the US. As we start trial two the associative strength is V is abL so the change in strength that occurs with the second pairing of the CS and US is ab(L - abL). It is smaller than the amount learned on the first trial and this reduction in amount that is learned reflects the fact that the CS now has some association with the US, so the US is less surprising. As more trials ensue, the equation predicts a gradually decreasing rate of learning which reaches an asymptote at L. However, the diagramm below shows: this is not what is seen when the development CS-US associations is measured over time. Instead the learning curve is sigmoidal. Rescorla has argued that the equation is consistent with observed behavior if one assumes that very small changes in associative strength are undetectable and that there is a limit to the amount of effect that very large changes can have on behavior. </P> <P><IMG SRC="acquisit.gif" Alt="CS-US aquisition"></P> <P>There are other respects, however, where the model performs better in predicting experimental outcomes. It can also be applied to a number of CSs each of which contributes to an overall associative strength V of the US in the right hand side of the equation. It is reasonably clear that the presence of the CS salience term b in the equation lets it account for overshadowing. The meaning of the equation is clearest if the specific dVs on the left hand side are seen as referring to the increments in association between specific CSs while V on the right hand side is referring to the predictability of the US and so is the sum of all the different CS-US associations. If the conditioning strength accrued to CS1 is denoted by dV1 and that to CS2 by dV2 then our equations are </P> <P><B>dV1 = ab1(L - V)</B></P> <P><B>dV2 = ab2(L - V)</B></P> <P>and both dV1 and dV2 accrue to V on each trial. The amount of association directed to each CS is proportional to their salience. </P> <P>The equation also models blocking well. During the initial phase of a blocking experiment the associative strength of the US is increased so later, when a second CS is presented the amount of associative strength it can gain has been reduced. </P> <P>The critical question is, however, does the model predict experimental outcomes it was not explicitly divised for, i.e. can it be generalized? In one example the model predicts the effects of pairing two previously learned CSs on learning about a third new stimulus. If on separate occasions (not as compound stimuli) two CSs of equal salience have both been completely associated with a US then V=L for both stimuli and dV on subsequent trials is zero for both. Now a third CS in conjunction with the original pair is presented so three CSs are presented together whereas only two of them were presented singly in the past. The overall associative strength of the US is now 2L, a contribution of L from both of the original CSs. The equation predicts that there will be a negative change in associative strength on this trial proportional to the salience of the CSs: </P> <P><B>dV = ab(L - 2L)</B></P> <P><B>dV = -abL </B></P> <P>Conducting the experiment shows: the third stimulus becomes a conditioned inhibitor of the US - it provokes a CR of the opposite quality to that produced by the other two CSs. </P> <P>Rescorla's explanation of the "blocking and predictability" experiment is more debatable. During phase 1 of the experiment the 'No US' are undergoing habituation. Rescorla argues that the 'No US' group learn in the first phase of the experiment that CS1 is a predictor of 'no US' and hence that, when it is followed by a US in phase 2 this US is even more surprising than it would have been normally, hence it provokes especially strong learning. His own model, however, predicts that there should be no change in the associative strength associated with the stimulus when there is no US. First, is is not very logical to assign an amount of processing devoted to a non-event if that non-event is unpredicted. Second, Rescorla's model revolves around the surprisingness of specific USs - and 'no US' must be a different US from 'US' so prior exposure to a good predictor of 'no US' should not effect the amount of processing devoted to a different US 'US'. For these, and other reasons a series of more sophisticated models have subsequently been developed in which the rate of learning is not driven by the 'surprisingness' of the US (as in the L-V term of the Rescorla-Wagner model) but by terms which represent the predictive power of individual CSs independently (for example Mackintosh's 1975 model). In this sort of model a CS which had been experienced many times unpaired with a significant US would be evaluated as having less than average predictive power. If, however, the CS had been paired with a different US during phase 1 of Rescorla's (1971) experiment, then it should be evaluated as having predictive power and hence still be associable with a different US during phase 2, reducing the 'superconditioning' to the other CS previously found. Dickinson (1976) has reported such an effect. </P> <H2>Where are associations made?</H2> <P>In the basic classical conditioning paradigm there are four components, US, CS, UR and CR. Although the CR and UR may come to differ markedly, the initial learned response to the CS is the UR. Through what association does the CS come to elicit the UR? The figure below shows: initially there is a link between the US and the UR but none from the CS to the US or UR. It is quite plausible that making either association would cause the CS to elicit the UR - during training the CS is a predictor of both the US and UR (since the UR inevitably follows the US). </P> <P><IMG SRC="ccassoc.gif" Alt="associations in classical conditioning"></P> <P>After conditioning the CS could evoke the CR through a direct stimulus response association or it could evoke the CR indirectly through an stimulus-stimulus association with the US. In the latter case, when the CS's association with the US evokes some trace of the US in the memory of an animal, this trace, in turn evokes the UR reflexively. In the former case the CS can effectively replace the US, this hypothesis is therefore sometimes known as stimulus-substitution - Pavlov believed this was the underlying mechanism of classical conditioning. Evidence exists to support the involvement of both types of association in classical conditioning. </P> <P><I>Sensory-preconditioning:</I></P> <table border> <td>Preconditioning<td>Conditioning<td>Test<tr> <td>CS2-CS1<td>CS1-US<td>CS2<tr> </table> <P>The phenomenon of sensory-preconditioning supports the existence of stimulus-stimulus associations. In sensory pre-conditioning two neutral stimuli which one might use as CSs are repeatedly presented together without any US. One of the stimuli, CS1, is now conditioned to a US so that it elicits a CR. If the other stimulus, CS2, is now presented to the animal in the absence of the US it too elicits a CR. The only explanation can be that the animal learned an association between CS1 and CS2 during the sensory preconditioning phase of the experiment, so clearly these stimulus-stimulus associations are learnable. One cannot, however, clearly infer from this result is that the animal must have learned a stimulus-stimulus association between CS1 and the US during the conditioning phase. Both CS2-CS1-CR and CS2-CS1-US-UR associations could explain the behavior observed. The process of second-order conditioning is, in some ways, a reversal of the sensory-preconditioning procedure. Here CS1 is paired with the US. Thereafter CS2 and CS1 are paired and once again test the subject's response to CS2 alone. CS2 elicits a CR and again it is unclear whether this can be attributed to stimulus-stimulus or stimulus-response associations. Adding a final twist to the experiment, however, provides more conclusive evidence for stimulus-response associations. </P> <P>Switching second-order conditioning USs:</P> <table border> <td>CS1-US1 CS2-CS1 CS1-US2 CS2-<td>?<tr> <td><td>?=UR1 implies S-R learning<tr> <td><td>?=UR2 implies S-S learning<tr> </table> <P>Having established the association between CS2 and CS1, now CS1 is paired with a different US we now finally test the response that CS2 elicits. If conditioning is primarily a matter of learning stimulus-response associations then CS1 will become associated with the response UR1 during the first phase of the experiment and a similar CS2-UR1 association will be produced in the next phase. When CS1 is paired with the new US2 this should not effect the response produced to CS2. If, on the other hand, associations are primarily made between stimuli then the response to CS2 should change when CS1 is paired with US2 through the chain of associations CS1-CS2-UR2. In fact UR1 is produced in the final test phase of the experiment, supporting a stimulus-response model of conditioning. Notwithstanding the evidence above, there is also some evidence that classical-conditioning cannot be based solely on stimulus- response associations. It has been found that interfering with an animal's ability to make a UR during conditioning (for example by temporarily paralysing the animal with curare so that muscular URs like leg-flexion could not occur, or temporarily inhibiting salivation with atropine and so blocking part of the typical appetitive US) does not eliminate its subsequent production of a CR in response to the CS. This is quite incompatible with a simple model of CS-UR association because there is no UR to associate the CS with during conditioning. One attempt to reconcile this result with the second-order conditioning results is to suggest that it is not the UR which becomes associated with the CS, but rather, the motivational state which to which the UR is directed. This explanation would hold that a CS becomes associated with hunger, not salivation, in an appetitive conditioning experiment and fear rather than cowering in conditioned emotional responses. </P> <H2>What are associations made between?</H2> <P>The notion that associations might not be made to responses but to the motivational states that normally provoke those responses, raises the question of whether associations in general are being made between events, that is, stimuli and responses, or between their representations in memory. Asserting that associations are made between appropriate motivational states is really just the first step towards asserting that associations are made between representations. Examining the development of CRs and the way they come to differ from URs provides evidence that associations are made between the motivational states associated with stimuli and responses, and not simply with the stimuli and responses themselves. As mentioned earlier, URs and CRs might differ. One part of the UR to a US of electric shock is an increase in heart-rate. In a well trained animals, however, the CR they make to a CS which predicts shock is a not an increase, but a decrease, in heart-rate. Similar effects are found in CRs conditioned to drug USs. For example, if a tone is repeatedly paired with administration of analgesic doses of morphine the CR to the tone alone is not the decrease in pain sensitivity produced by morphine, but an increase in pain-sensitivity which might be thought of as compensating for the analgesic effects of morphine. These antagonistic CRs certainly fit in with a model of conditioning in which the motivational states associated with stimuli, rather than the stimuli themselves, are associated. Other experiments on blocking indicate that more general representations of the qualities of stimuli are being associated. Bakal, Johnson & Rescorla (1974) added a third group of subjects to a standard blocking experiment, who, rather than being conditioned with the same US in both training phases were conditioned with two different USs, both aversive. This change of USs had little influence on blocking, both groups failed to learn an association with CS2 during the second phase of the experiment. The explanation proffered was that, rather than learning that CS1 predicts a specific type of aversive event, it became a good predictor of aversive events in general. </P> <P>Bakal, Johnson & Rescorla's (1974):</P> <table border> <td>Phase 1<td>Phase 2<td>Test<tr> <td><td>CS1-CS2-Klaxon<td>CS2<tr> <td>CS1-Klaxon<td>CS1-CS2-Klaxon<td>CS2<tr> <td>CS1-Shock<td>CS1-CS2-Klaxon<td>CS2<tr> </table> <P>One might argue that this is not so different from arguing that an association has been made between CS1 and the motivational state of fear which both USs (the klaxon and the shock) evoked. A refinement of this design by Dickinson strengthens the case that associations are made with quite general properties which form parts of the representation of events rather than specific motivational states. Dickinson trained animals to expect a positive event to happen regularly (receiving some food: US+) and then showed that associating a CS with the omission of this expected nice event (CS1 in group 'a' signals that CS3 will not be followed by US+) could block learning when this CS was paired with a second CS (CS2) and an aversive US (US-) during the second phase of the experiment. The appropriate controls are ones in which CS1 is experienced with postive consequences in phase 1 (group 'b') and where CS1 has never been experience before phase 2 (group 'c'). </P> <P>Dickinson & Dearing's 1979 experiment:</P> <table border> <td>group<td>Phase 1<td>Phase 2<td>Test<tr> <td>a<td>CS3 US+<br> CS3 CS1 <td>CS1 CS2 US-<td>CS2<tr> <td><td>CS1 implies no US+<td><td><tr> <td>b<td>CS3 US+<br> CS1 US+<td>CS1 CS2 US-<td>CS2<tr> <td><td>CS1 implies US+<td><td><tr> <td>c<td>CS3 US+<td>CS1 CS2 US-<td>CS2<tr> <td><td>CS1 is unpredictive<td><td><tr> </table> <P>This result is very hard to explain in terms of association with a specific motivational state, but quite straightforward assuming that CS1 has been associated with events having bad consequences in general. </P> <FONT SIZE="-3" COLOR="#FF0000">This article is restructured from a lecture kindly provided by <A HREF="http://fnord.dur.ac.uk/~dps1rwk/">R.W.Kentridge</A>.</FONT>