Operant reward learning in Aplysia: Neuronal
correlates and mechanisms
Björn Brembs, Fred D. Lorenzetti, Fredy D. Reyes, Douglas A. Baxter,
John H. Byrne
Aplysia biting behavior. The consummatory phase of Aplysia
feeding behavior (biting) occurs in an all-or-nothing fashion
and is unambiguously quantifiable (1, 2). It consists
of four phases: jaw opening, odontophore/radula protraction in
the open state, odontophore/radula retraction in the closed state
and jaw closure. Biting occurs spontaneously as well as reflexively,
in nature as well as in the laboratory. If food is present, it
leads to the ingestion of food through the buccal cavity. In this
8 s video sequence, the animal has positioned itself under the
water surface (as they often do). Its tentacles (anterior) are
at the top of the screen. The sequence contains the opening of
the jaws, followed by the protraction of the radula (cream colored
tongue-like organ) in the open state, the closure of the radula,
the retraction of the radula in the closed state and the closing
of the jaws.
To view this video, download a QuickTime
Quicktime video: 176x144
pixels (123,925 bytes) | 800x600
pixels (2,602,794 bytes)
Supplemental Methods 1.
Surgical procedures and in vivo recordings. Aplysia
californica (100-200 g) were obtained from Alacrity Marine
Biological Specimens (Redondo Beach, CA) and Marinus (Long Beach,
CA). They were kept individually in rectangular perforated plastic
cages floating in aerated artificial seawater (Instant Ocean;
Aquarium Systems, Mentor, OH) at a temperature of 12-15°C. Animals
were fed ~1 g of dried laver, 3 times a week. To help ensure that
all animals were in a similar motivational state, experimental
animals were food deprived 3-5 days before surgery.
Extracellular electrodes were prepared from 25.4 µm insulated
stainless steel wire (304 TRI-ML, California Fine Wire Co., Grover
Beach, CA) by scraping the insulation from 2 mm at one end of
a piece (~30 cm) of wire and from about 15 mm at the other end.
Two such wires were used, one for recording/stimulating, the other
for reference. The two long de-insulated ends were soldered to
gold contacts, whereas one of the short de-insulated ends was
fashioned into a small hook for placement on the nerve (for stimulation/recording).
The other wire served as the reference electrode and its tip was
Just prior to surgery, the animals were fed a single bite of seaweed,
to probe the motivational state, overall feeding behavior and
health of the animals. Animals that did not feed were discarded.
Each animal was then anaesthetized by injecting isotonic MgCl2
(360 mM) solution (30% body weight) into the hemolymph and transferred
onto a block of ice (made of seawater), covered with artificial
seawater. The animal was positioned with its left body side up
and two hooks shaped from hypodermic needles (30 G1/2; Becton
Dickinson, Franklin Lakes, NJ) were placed in the skin ~3 cm apart
~1.5 cm ventral of the eye and in parallel with the anterior-posterior
axis of the animal. The hooks were fastened by threads on either
end of the tank and tightened so as to lift the fold of skin between
them above the water surface. A small (1-1.5 cm) incision was
made between the hooks alongside the fold and with the eye being
at the height of the anterior third of the incision. During the
surgery, the thread suspended hooks were used to keep the wound
above the water level to prevent leakage of hemolymph, or seawater
from entering the animal. The incision was kept open using a second
pair of hooks arranged perpendicularly to the first pair. A moveable
support was used to lift the buccal mass and expose the buccal
ganglia and their nerves. Additional manipulators were then used
to place the hook-electrode around the anterior branch of the
esophageal nerve (En2), close to the anterior/posterior
branch point. The electrode was secured and insulated by a drop
of superglue (Loctite Quick Gel, part # SGG-2B, Loctite, Rocky
Hill, CT). After inserting the reference electrode to float free
in the hemocoel, the animal was closed using 4-6 stitches of 4-0
black braided silk suture (Ethicon, Somerville, NJ), with the
electrodes exiting the animal at the posterior end of the incision.
The entire procedure lasted between 45-60 minutes for each animal.
After the surgery, the animals were placed into individual rectangular
perforated plastic cages and left to recover overnight at 12-15°
C. Electrode signals were amplified using a differential AC amplifier
(model 1700; A-M Systems, Everett, WA), filtered by a 100 Hz low
cut-off filter and a 1 kHz high cut-off filter. One day after
surgery, extracellular activity in the anterior branch of the
esophageal nerve was recorded while the animal was observed in
a round glass bowl (radius: 80 mm, depth: 70 mm) placed on a mirror.
During the observation period, the animal was stimulated to bite
and swallow with pieces of seaweed. The animals were not restrained
and the length of the wires permitted a full range of motion in
the bowl. Animals used for in vivo recording were not used
in the conditioning studies.
Supplemental Methods 2.
Operant reward learning. The occurrence of spontaneous
bites is dependent on a number of variables. While it can be observed
in freshly caught specimens, it is comparatively rare. To successfully
conduct experiments, certain measures need to be taken to increase
the frequency at which this behavior occurs. To help ensure that
all animals were in a similar motivational state, experimental
animals were food deprived 3-5 days before surgery. One day after
implanting the stimulating electrodes, the animals were fed a
single bite of seaweed 30 minutes before the experiment to motivate
the animal to search for food. Seaweed extract was prepared by
incubating a 10 cm x 20 cm piece of seaweed in 300 ml of artificial
seawater for 30 minutes. Pilot studies found seaweed extract to
increase the overall probability of biting behavior to occur.
Just prior to the experiment, 50 ml of the supernatant were added
to 400 ml of fresh artificial seawater. The animal was then transferred
into a round glass bowl (radius: 80 mm, depth: 70 mm) containing
these 450 ml of diluted seaweed extract and the bowl placed on
a mirror to be able to better observe the animal. The experiment
was performed in a climate chamber at 15° C and 60% rel. humidity.
Unrestrained in the bowl, the animal moved around freely and engaged
in spontaneous behaviors (A). Throughout the experiment, the animal
was observed and all bites recorded. A bite (see supplemental
video) was defined as opening of the jaws and protraction of the
radula. Before the start of the experiment, animals were assigned
to one of three groups (B): i) a control group that did not receive
any stimulation, ii) a contingent reinforcement group which received
En2 stimulation whenever the jaws closed after a bite
during training, or iii) a yoked control group that received the
same sequence of stimulations as the contingent group, but the
stimulation occurred uncorrelated with their behavior. Except
for the different reinforcement schedules, all animals in all
three groups were treated identically. Application of reinforcement
was the only difference between training and test. In the early
phase of the study, the animals were assigned randomly to each
group. As the study progressed, the animals were assigned to each
group so as to balance pre-test bite rate between groups. Experimental
sessions consisted of four consecutive five-minute periods. In
each period, the number of bites was recorded. The final test
period was either immediately after training (I-Test) or 24 h
after the beginning of the experiment (L-Test). A Grass S48D stimulator
(Grass Instruments, Quincy, MA) generated 10 ms pulses for extracellular
nerve stimulation (30 Hz, 3 s). Pilot studies determined that
a suitable intensity of the stimulation was 8 V. At this voltage,
usually no behavioral response could be observed. Occasionally,
an animal (mostly yoked controls) would show a jaw opening without
radula protraction or a rejection-like behavior (i.e., the radula
appeared to be closed during protraction) to the first few stimulations
only. Such responses were never observed spontaneously. If they
met the definition of a bite (i.e., the radula was protracted),
they were scored as bites irrespective of the subjective impression
of the observer. Animals that were tested after 24 h spent the
time between training and test in individual rectangular perforated
plastic cages in aerated artificial seawater at 12-15° C. On the
next day, the animals were placed back into the glass bowl with
seaweed extract, but without being fed before the test. After
the experiments, all animals were sacrificed and electrode placement
verified. Only animals that produced 0 < n < 31 bites in the pre-test
were used. Several experimenters independently replicated this
experiment and all their data were pooled.
Medium version | Full
Supplemental Methods 3.
Biophysical correlates of the operant memory in B51. Animals
from a second behavioral study were anesthetized by injecting
a volume of isotonic MgCl2 equivalent to 50% of the
animal's weight. Buccal ganglia were removed and pinned on a Sylgard-coated
Petri dish containing artificial seawater (ASW). The composition
of the ASW was: 450 mM NaCl, 10 mM KCl, 30 mM MgCl2(6H2O),
20 mM MgSO4, 10 mM CaCl2(2H2O),
10 mM HEPES, with pH adjusted to 7.4. The ganglion ipsilateral
to the esophageal nerve stimulation was desheathed on the rostral
side. Desheathing was performed in the presence of high divalent
cation ASW solution, which contained concentrations of CaCl2
and MgCl2 that were three times the normal level. Osmolarity
was maintained by correspondingly decreasing the concentration
of NaCl. After desheathing, the medium was changed to normal ASW.
Neuron B51 was identified based on its relative size and position
in the ganglia. The identity of the cell was confirmed by its
pattern of electrical activity during a buccal motor program,
membrane properties, and its characteristic plateau potential
as described in (3,4). Recording temperature was 15° C.
Conventional two-electrode current-clamp techniques were used
for intracellular recordings (Axoclamp-2A, Axon Instruments, Burlingame,
CA). Fine-tipped glass microelectrodes (resistance 10-15 MOhms)
were filled with 2 M potassium acetate. The cell was current-clamped
at -60 mV for the duration of the experiment. Five minutes after
impalement with the electrodes, the input resistance and the burst
threshold were measured. If a spontaneous motor pattern occurred
during testing, the test was halted and then resumed 1 minute
after the cessation of the pattern. The input resistance of B51
was determined by injecting a hyperpolarizing current of 5 nA
for 5 s. The burst threshold of B51 was defined as the minimum
amount of depolarizing current necessary to elicit activity in
B51 that outlasted the current pulse. The burst threshold was
tested by a series of successively higher amplitude depolarizing
current pulses (in 1 nA increments) with a duration of 5 s. The
series was spaced with 10 s between the end of one pulse and the
start of another. In this way the minimum amount of current necessary
to elicit a plateau potential could be determined. After these
two properties were measured, the cell was released from current
clamp and the resting membrane potential was determined. On average,
membrane properties of B51 were recorded 100 minutes after the
last training period. The experimenter performing the intracellular
measurements was unaware of the experimental history of the animals.
Supplemental Methods 4.
B51 cell culture and electrophysiology. Culturing procedures
followed those described in (5-8). Buccal ganglia from
adult Aplysia were incubated in 1% protease type IX (Sigma,
St. Louis, MO) at room temperature for 24 hours and then desheathed.
In pilot studies, B51 neurons in the buccal ganglia were first
identified by the electrophysiological methods described previously
and then dye-labelled with Fast Green (Sigma). B51 neurons were
removed from the ganglia by microelectrodes with fine tips and
plated on poly-L-lysine coated glass slides in petri dishes with
culture medium containing 50% hemolymph and 50% isotonic L15 (Sigma).
The cells were allowed to grow for 4-5 days and the medium was
changed on the third day. Culture medium was exchanged for ASW
prior to recording. It was found that neurite morphology coupled
with the size and the relative position of the cell in the ganglia
was sufficient to identify B51. Thus, these criteria were adopted
as the means of identification for all the neurons used in this
The electrophysiological methods used to record from cultured
neurons were largely the same as those used to record from neurons
in the ganglia. Due to the high input resistance in cultured cells,
the cells were current clamped to -80 mV. Five minutes after impalement,
input resistance and burst threshold were determined. Input resistance
was tested by injecting a hyperpolarizing current pulse of 0.5
nA for 5 s and burst threshold was tested in 0.1 nA increments.
The cells were then divided into a contingent reinforcement and
an unpaired group. Plateau potentials were generated by a 5 s
long depolarizing current pulse with an amplitude 0.1 nA higher
than the previously determined threshold. Both groups received
7 evenly spaced supra-threshold depolarizing current pulses in
a ten-minute training period. The cells in the contingent reinforcement
group received a 6 s iontophoretic pulse of dopamine immediately
after the cessation of the plateau potential, whereas iontophoresis
was delayed by 40 s in the unpaired group. Dopamine was iontophoresed
through a fine-tipped glass microelectrode (resistance 10-15 MOhms).
A retaining current of -1 nA was used during the course of the
experiment. A square wave current pulse of 35 nA for 6 s was used
to eject the dopamine. The concentration of dopamine in the electrode
was 200 mM. An equimolar concentration of ascorbic acid was added
to the electrode to reduce the oxidation of dopamine. After training,
the membrane properties were measured again and compared to the
Recordings were performed at room temperature.
From postural adaptation to social interaction, operant conditioning
is one of the essential processes leading to the generation and
modulation of behavior. However, its analysis has been complicated
because most learning situations inseparably comprise operant
and classical components. Specifically, behaving organisms constantly
receive a stream of sensory input that is both dependent and independent
of their behavior. The classic debate as to whether one or two
processes account for the operant/classical dichotomy reflects
this entanglement (e.g., 9-14). Interrupting the operant
feedback loop by restraining an animal can at least partly isolate
classical conditioning from the operant components. Once isolated
from spontaneous behavior, in a number of systems the stimuli
have been traced into the nervous system to find the point of
convergence where the classical association is formed. Until now,
the convergence of reinforcement and the operant behavior has
remained elusive, however.
Similar to the successful isolation of the stimuli from spontaneous
behavior in classical conditioning, the development of the present
procedure is a critical step towards operant conditioning without
any external stimuli other than the reward being contingent upon
the behavior (i.e., ‘pure’ operant conditioning, 14).
So far, it can not be ruled out that the animal can perceive the
sound and vibrations associated with recording the sequence of
stimulations and activating the stimulator. However, given the
nature of the sensory organs in Aplysia, this appears highly
unlikely. Thus, in practical terms, the bite occurs spontaneously
and except for the reinforcement, all external stimuli are either
constant or independent of the biting behavior. The nature of
the reinforcement also rules out classical contamination by predictive
‘internal’ stimuli generated by other types of reinforcement.
Whereas other reinforcers like food or water need consummatory
behavior (preceding the reward) to be effective, stimulation of
En2 is not preceded by any other behavior or stimuli
(external or internal) other than the rewarded operant behavior.
It cannot be ruled out that internal stimuli are generated by
the innervation of the buccal musculature and the salivary gland
by En2. However, such stimuli would be sensed after
the onset of the reward and can therefore not acquire any predictive
properties other than that they are not followed by reinforcement.
Contextual cues are always present during the experiment. Indeed,
without contextual cues, the association would most likely have
extinguished in the home tank before the 24 h test. However, these
cues were constant throughout the experiment and thus are not
predictive as to when exactly the reinforcement will occur (such
as a classical conditioned stimulus would). Therefore, it is safe
to contend that any contextual cues act as ‘occasion setters’
to indicate whenever the behavior – reinforcer relation is true
and do not directly enter into the primary operant association.
With the development of in vivo and in vitro classical and operant
procedures in Aplysia, the first detailed mechanistic comparison
between operant and classical conditioning in the same preparation
is within reach. Ultimately, the tools now available in Aplysia
will allow for studies of operant-classical interactions (e.g.,
References and Notes:
- I. Kupfermann, Behav. Biol. 10, 1 (1974).
- I. Kupfermann, Behav. Biol. 10, 89 (1974).
- M. R. Plummer, M. D. Kirk, J. Neurophysiol. 63,
- R. Nargeot, D. A. Baxter, J. H. Byrne, J. Neurosci.
19, 2247 (1999).
- S. G. Rayport, S. Schacher, J. Neurosci. 6,
- S. Schacher, E. Proshansky, J. Neurosci. 3,
- J. Chin, A. Angers, L. J. Cleary, A. Eskin, J. H. Byrne, Learn.
Mem. 6, 317 (1999).
- F. D. Lorenzetti, D. A. Baxter, J. H. Byrne, paper presented
at the 30th Annual Meeting of the Society for Neuroscience,
New Orleans, La., 7. November 2000.
- B. F. Skinner, J. Gen. Psychol. 12, 66 (1935).
- R. A. Rescorla, R. L. Solomon, Psychol. Rev. 74,
- I. Gormezano, R. W. Tait, Pavlov. J. Biol. Sci. 11,
- J. G. Holman, N. J. Mackintosh, Q. J. Exp. Psychol.
33B, 21 (1981).
- R. A. Rescorla, J. Exp. Psychol. Anim. Behav. Process.
20, 44 (1994).
- B. Brembs, M. Heisenberg, Learn. Mem. 7, 104