The Importance of Being Active

Abstract: The successful stimulus-response approach to the organization of behavior has been the dominating paradigm for much of the psychology and neuroscience of the 20^th century. Martin Heisenberg is a pioneer in championing the idea that all brains, even comparatively simple ones such as those of insects, instead operate according to output/input principles. Since the 1970s, his research produces evidence that the fruit fly, Drosophila melanogaster, is capable of spontaneous behavioral activity and that the flies use it to control sensory input (operant behavior). Today, more and more evidence is accumulating also from fields outside of neuroscience that indeed one of the common, defining principles of all brains is this concept of operant behavior. Drawing from this evidence, it becomes clear that the conceptually simple process of generating activity and evaluating its consequences forms one of the fundamental cornerstones not only for all of our human nature, but also for our social coherence. This review recapitulates Heisenberg’s most critical experiments and provides an overview over the current literature on the role of spontaneous activity in the ecology and evolution of brains. I conclude that spontaneous activity is both a necessary prerequisite and an inevitable consequence of evolution.

Probably one of the – if not the best-understood sensory system is the fly visual system. Over the last 60 years or so, on all levels of complexity, the systems, the single cell and more recently even the molecular level, neuroscientists have developed an understanding that is unparalleled both in breadth and in depth. The groundwork was laid by the early works of Götz, Reichardt and other colleagues in the tradition of biological cybernetics (Götz, 1964, 1965, 1968, 1972, 1977, 1980; Götz & Buchner, 1978; Götz, Hengstenberg, & Biesinger, 1979; Kirschfeld & Reichardt, 1970; Poggio & Reichardt, 1973a, 1973b, 1976a, 1976b; Reichardt, 1962; Reichardt & Poggio, 1975, 1976; Reichardt & Wenking, 1969; Reichardt, 1965; Wehrhahn & Reichardt, 1973). This tradition entailed to study the visual system with the tools of control theory. In principle this meant interpreting such experiments as manipulating a complex input/output system. The idea behind this very successful black-box approach was to study the input/output relationships thoroughly enough to be able to construct a control model which could predict the motor output of the fly for any, even yet untested visual input. One method of choice was often the so-called open-loop experiment in which the tethered fly (Fig. 1) received visual input while its motor output was recorded. Importantly, the motor output was not allowed to interfere with the presentation of the stimuli, i.e., the feedback loop between the animal’s behavior and its environment was open. This was the time when the young geneticist Martin Heisenberg joined this exciting field after his postdoctoral period with Seymour Benzer at CalTech. Heisenberg’s contributions to vision in Drosophila are covered elsewhere in this issue, so I will not go into any detail here (Wolf & Yamaguchi, this issue). This input/output approach worked extremely well, not only for biological cybernetics of visual guidance of insect flight. Many other areas of neurobiology at the time also thought of brains as input/output systems and prevented the behavior of their animals to interfere with their stimulus situation. In fact, so successful was this pervasive approach that until very recently some neurobiologists still emphasized that “brain function is ultimately best understood in terms of input/output transformations and how they are produced” (Mauk, 2000). This was the dominant tradition in which Heisenberg was working when he moved to Würzburg to become the chair of the department of genetics in 1975.

In Würzburg, several discoveries prompted Heisenberg to radically change his view on brains. Three of these in particular were instrumental for his 180° turn. First was the observation that even without any variation in sensory input, the flies would produce varying motor output (Fig. 2, Heisenberg & Wolf, 1979). This finding flew in the face of every control theory thus far. A later mathematical analysis of the temporal structure of the flies’ behavior in this situation (Maye, Hsieh, Sugihara, & Brembs, 2007) later confirmed Heisenberg’s interpretation that the variability in the behavior of the fly does not stem from a very noisy input/output system but was generated actively by the fly, independently of any input (i.e., “initiating activity”, Heisenberg, 1983, 1994). Importantly, Maye et al. (2007) found the same kind of variability in the behavior even when the flies were allowed to control their environment with the feedback loop closed. The second discovery was made with the double mutant reduced optic lobes (rol), small optic lobes (sol) (Fig. 3). Freely walking or flying wildtype flies in a visually structured rotating environment have a tendency to turn with the direction of the movement. The rol sol double mutant flies still show phototaxis, but are completely devoid of any such directed ‘optomotor response’. The optomotor response was thought to be critical for stabilizing the animal in flight and thus rol sol flies were expected to lack the capacity to use moving visual stimuli for course control and thus should show unstable flight. However, in experiments with tethered flies where the feedback loop between turning behavior and horizontal rotation of the environment was closed, rol sol mutant flies were able to stabilize their flight with respect to visual landmarks and fly straight (i.e., establish optomotor balance, Wolf & Heisenberg, 1986). The interpretation was that rol sol mutant flies are motion sensitive but lacked sensitivity to the direction of motion. This was demonstrated by performing the third critical experiment (Fig. 4). After inversion of the feedback loop between behavior and environment such that attempted left-turns lead to a left-turn of the environment and thus the visual impression of a right turn, rol sol mutants did not require any more time to stabilize their flight and fly straight than when then loop was closed ‘correctly’. The conclusion that flies are actively initiating activity in order to ‘try out’ which motor output controls the environment was confirmed when wildtype flies were subjected to this “inverting goggles” experiment. Even wildtype flies, with their optomotor response intact, eventually learned to use turning maneuvers of the ‘opposite’ direction to control flight, i.e. left turning maneuvers for the visual impression of right turns and vice versa (Heisenberg & Wolf, 1984; Wolf, Voss, Hein, & Heisenberg, 1992). These three experiments attacked contemporary control theory from two ends: not only was output not predictable from input, but eliminating the open loop response or inverting its direction could be compensated by plasticity in the system such that it would still perform the function in question. Not only were the open loop situations an inadequate experimental approach, the input/output assumption itself proved inadequate as a theoretical construct for understanding even a fly brain. These experiments prompted Heisenberg to abandon this pervasive and successful approach and pursue a radically different research direction: how animals use their capacity for initiating output to control their sensory input. Today, thirty years later, more and more evidence not only from neuroscience is accumulating that while computing input/output relations may be an important feature of brains, computing such output/input relations (e.g., in so-called forward models, Webb, 2004) is probably equally, if not more fundamental to the organization of behavior. In the remainder of this article, I will try to provide a short sketch of the literature today is supporting Heisenberg’s early insight.

If we allow ourselves to anthropomorphize, Heisenberg’s observation that flies with a constant stimulus situation still produce variable behaviors may not be all that surprising. We have all experienced how difficult it is to stay absolutely still, maybe even how horrible it feels to be forced not to move. We want to move and we want to decide what body-part to move when and where the movement should go. We all feel the very basic notion that we possess a certain flexibility in our choices (Montague, 2008). Bereaving humans of such freedom is frequently used as punishment and the bereft do invariably perceive this limited freedom as undesirable. This experience of freedom is an important characteristic of what it is like to be human. It stems in part from our ability to behave variably. Voltaire expressed this intuition in saying “Liberty then is only and can be only the power to do what one will” (Voltaire, 1752/1924). But the concept that we can decide to behave differently even under identical circumstances underlies not only our justice systems. Electoral systems, our educational systems, parenting and basically all other social systems also presuppose behavioral variability and at least a certain degree of freedom of choice. Games and sports would be predictable and boring without our ability of constantly changing our behavior in always the same settings. Faced with novel situations, humans and most animals increase their behavioral variability (Bunzeck & Duzel, 2006; Roberts & Gharib, 2006; Shahan & Chase, 2002). Animals even vary their behavior when a more stereotyped behavior would be more efficient (Krechevsky, 1937). Inasmuch as behavioral variability between individuals has genetic components, it is a crucial factor of niche exploitation in evolution. Moreover, behavioral variability within individuals has been shown to be ecologically advantageous in game theoretical studies (Brembs, 1996; Glimcher, 2003; Glimcher, 2005; Glimcher & Rustichini, 2004; McNamara, Barta, & Houston, 2004; Platt, 2004), in pursuit-evasion contests such as predator/prey interactions ("Protean Strategy") (Driver & Humphries, 1988; Grobstein, 1994; Miller, 1997; Shultz & Dunbar, 2006), in exploration/foraging (Belanger & Willis, 1996; Viswanathan et al., 1999), in mobbing attack patterns by birds and in the variation of male songbirds’ songs (Neuringer, 2004). Clearly, invariable behaviour will be exploited (Jablonski & Strausfeld, 2000, 2001; Miller, 1997) and leaves an organism helpless in unpredictable situations (Greenspan, 2005; Heisenberg, 1994).

Thus, competitive success and evolutionary fitness of all ambulatory organisms rely critically on intact behavioral variability as an adaptive brain function. But relative freedom from environmental contingencies is a necessary, but most often not a sufficient criterion for such accomplishments. Tightly connected to the ability to produce variable behavior is the ability to use the effects of these behaviors to control the environment. The incoming stream of sensory information is noisy and fluctuates for any number of reasons. Any covariance between the behavioral variations and those of sensory input indicates that the latter are consequences of the behavior and can thus be controlled be the animal (Bays, Flanagan, & Wolpert, 2006; Wegner, 2002). Every animal relies heavily on this on-line detection system for when the animal itself is the reason for any environmental fluctuation. This function is so paramount, that we humans express our delight over control of our environment (including other people) already as children, by e.g., shrieking in excitement when Dad jumps after a “boo” or proudly presenting Mom with “look what I can do!”. Later, children find pleasure in building airplane models, become carpenters with a delight for shaping wood, artists feeling gratified creating art out of the simplest materials, musicians enjoying mastering their instrument to perfection, athletes, scientists, engineers, managers or cunning politicians. Using trial and error, we have shaped our world from caves to skyscrapers, from horses to jet-planes, from spears to hydrogen bombs. Cultural or religious rituals (e.g., rain dance) and superstition may have developed as means to create a feeling of control where ultimately there is none. Clearly, behaving flexibly in order to control our environment is at the heart of human nature and probably affects more aspects of our daily lives than any other brain function. So essential is such functioning that even very simple brains possess it. Even Drosophila prefers a situation in which it controls its environment over one where it does not. If certain flight directions are experimentally superimposed with uncontrollable visual movements, flies quickly avoid such directions and fly only in areas of full control (Heisenberg, Wolf, & Brembs, 2001). This experiment demonstrates that control over environmental stimuli is inherently rewarding already for numerically simple, but very likely also for all other brains. The same experiment also helps to understand how the rol sol mutants managed to fly straight: the flies spontaneously varied their motor output (“trying out”) until they could control absolute movement (i.e., independently of direction) in their environment. Obviously, rol sol mutants are using an operant strategy to control their stimulus situation (Wolf & Heisenberg, 1986). The same strategy also must underlie the capacity of wildtype flies to master the “inverting goggles” experiment.

By detecting what component of the sensory stream is controlled by our own actions, operant behavior also underlies the distinction between observing and doing, i.e. differentiating between self and non-self. It is thought that one of the main principles behind operant behavior is the so-called reafference principle (Todorov, 2004; von Holst & Mittelstaedt, 1950; Webb, 2004). We compare our behavioral output (efference) with incoming sensory input (afference) to detect when we are the ones authoring environmental change (Bays et al., 2006; Wegner, 2002; Wolf & Heisenberg, 1991). One almost iconographic example of such behavior is to perform various spontaneous movements in front of a mirror to detect whether it is us we are perceiving. Even animals perform such movements in front of a mirror (Plotnik, de Waal, & Reiss, 2006; Prior, Schwarz, & Güntürkün, 2008; Reiss & Marino, 2001). This automatic detection-mechanism explains why we cannot tickle ourselves (Bays et al., 2006), why we perceive a stable visual world despite our frequent quick, or saccadic, eye movements (Sommer & Wurtz, 2006) and is reflected in different brain activation patterns between self-generated and exogenous visual stimulation (Matsuzawa, Matsuo, Sugio, Kato, & Nakai, 2005). It is thought that the detection is accomplished via an efference copy (or corollary discharge) of the motor command which is compared to incoming afferent signals to distinguish re-afference from ex-afference. Such a differentiation has been implied to demonstrate causal reasoning in rats (Blaisdell, Sawa, Leising, & Waldmann, 2006; Clayton & Dickinson, 2006; Waldmann, Hagmayer, & Blaisdell, 2006). Even robots can use such “self-modeling” to generate a continuously updated model of themselves and their environment (Bongard, Zykov, & Lipson, 2006). The brain then uses this model to predict the sensory consequences of behavior and the integration of this prediction with the actual sensory information is used to produce an estimate of sensory space that is enhanced over predictions from either ex-afferent or re-afferent stimulation alone (Vaziri, Diedrichsen, & Shadmehr, 2006). This effect of operant enhancement of sensory cues can be observed also in fruit fly (Brembs & Plendl, 2008; Heisenberg et al., 2001), monkey (Kornell & Terrace, 2007), human (James, 1890; Slamecka & Graf, 1978) and robot (Gutnisky & Zanutto, 2004b) learning and may explain why starlings, but not tamarin monkeys can recognize patterns defined by so-called recursive grammar (Marcus, 2006). Such control of sensory input has often been termed “goal-directed” behavior or action. At its basis lies the capacity to generate spontaneous variability: initiating activity. This perspective provides an intuitive understanding of the rewarding properties of being in control of the environment. Setting and obtaining goals is inherently rewarding (Kim, Shimojo, & O'Doherty, 2006). This reward ensures that individuals always actively strive to control. Expecting sensory feedback signals can go so far that willing to move a limb can lead to the illusion of limb movement, even if none occurred (Gandevia, Smith, Crawford, Proske, & Taylor, 2006). One may also say that we so want our actions to have an effect that we sometimes develop a bad conscience even when we have not done anything wrong.

At the same time, by controlling the environmental input using operant feedback loops, individuals exert their effect not only on themselves, but their survival and procreation in the environment they shape for themselves directly affects evolution. This has been shown in the field, e.g., for western bluebirds, which dissociate into different niches according to their level of aggression (Duckworth, 2006). In humans such mechanisms have been proposed to explain otherwise hard to understand phenomena such as high IQ heritability estimates and associated paradoxa (i.e., increasing IQ heritability with age/experience and the “Flynn-Effect” of increasing IQ over generations) (Dickens & Flynn, 2001; Toga & Thompson, 2005). Another good example is the evolution of brain size. Most inter- and intraspecific interactions can be conceptualized as pursuit/evasion contests (e.g. predator/prey, male/female, dominant/subordinate etc.). There are two reports on such contests leading to increased brain size. The first details how small-brained prey are more likely to be caught by predators, presumably because their capacity for behavioral variability is also smaller (Shultz & Dunbar, 2006). The second shows that the largest relative brain sizes among primate species are associated with monogamous mating systems, raising the suspicion that unpredictable mating strategies are the most successful ones in monogamous species (Schillaci, 2006). Other research in birds ties the evolution of brain size both to behavioral variability and migration: birds with larger brains are both more likely to be sedentary and cope better in novel environments. The hypothesis here is that a sedentary lifestyle in seasonally changing habitats requires significant behavioral flexibility. Operant feedback provides flexible birds with more resources which enable them to support larger brains which in turn generate more behavioral variability: Brain size and behavioral flexibility co-evolved to out-compete other, smaller-brained birds which migrate in order to survive (Pravosudov, Sanford, & Hahn, 2007; Sol, Duncan, Blackburn, Cassey, & Lefebvre, 2005; Sol, Lefebvre, & Rodriguez-Teijeiro, 2005). Thus, the interdependence of brain size, the level of behavioral variability it provides and the energy supply by which it is constrained are starting to unravel.

With this short overview I hope to have shown that without initiating activity there would not be any brains for us to study. Such spontaneous activity is both a necessary prerequisite and an inevitable consequence of evolution. Ultimately, the conceptually simple process of generating activity and evaluating its consequences forms one of the fundamental cornerstones not only for all of our human nature, but also for our social coherence: human nature as described in planning, willing and controlling our behavior (Frith, Friston, Liddle, & Frackowiak, 1991; Knight, Grabowecky, & Scabini, 1995; Lezak, 1995; Owen, 1997; Wegner, 2002) and our social coherence as based on cooperation (Gutnisky & Zanutto, 2004a; McNamara et al., 2004; Sanabria, Baker, & Rachlin, 2003). Martin Heisenberg is a visionary and a pioneer in the neurobiological study of these and related neural processes. Only recently has his view of the primarily active nature of brains started to gain a more widespread acceptance among biologists.

References

Bays, P. M., Flanagan, J. R., & Wolpert, D. M. (2006). Attenuation of Self-Generated Tactile Sensations Is Predictive, not Postdictive. PLoS Biology, 4(2), e28.

Belanger, J. H., & Willis, M. A. (1996). Adaptive control of odor-guided locomotion: Behavioral flexibility as an antidote to environmental unpredictability. Adaptive Behavior, 4(3-4), 217-253.

Blaisdell, A. P., Sawa, K., Leising, K. J., & Waldmann, M. R. (2006). Causal Reasoning in Rats. Science, 311(5763), 1020-1022.

Bongard, J., Zykov, V., & Lipson, H. (2006). Resilient Machines Through Continuous Self-Modeling. Science, 314(5802), 1118-1121.

Brembs, B. (1996). Chaos, cheating and cooperation: Potential solutions to the Prisoner's Dilemma. Oikos, 76(1), 14-24.

Brembs, B., & Plendl, W. (2008). Double dissociation of PKC and AC manipulations on operant and classical learning in Drosophila. Current Biology, 18(15), 1168-1117.

Bunzeck, N., & Duzel, E. (2006). Absolute Coding of Stimulus Novelty in the Human Substantia Nigra/VTA. Neuron, 51(3), 369-379.

Clayton, N., & Dickinson, A. (2006). Rational rats. Nat Neurosci, 9(4), 472-474.

Dickens, W. T., & Flynn, J. R. (2001). Heritability estimates versus large environmental effects: the IQ paradox resolved. Psychol Rev, 108(2), 346-369.

Driver, P. M., & Humphries, N. (1988). Protean behavior:  The biology of unpredictability. Oxford, England: Oxford University Press.