Computational behavior theory and cultural evolution
Clarck’s nutcracker (image source)
Animal memory surprises us in many ways. How come, for instance, that Clarck’s nutcracker (pictured) can remember the location of thousands of seeds for many months, but cannot remember the color of a light for more than thirty seconds? To make sense of this and similar pradoxical findings, in a new paper we look at the performance of different species in the delayed matching-to-sample task (DMTS). This somewhat unwieldy name stands for a very simple procedure: we show a sample stimulus for a few seconds, then take it away a wait for a delay. At the end of the delay we show two stimuli: one identical to the sample, the other different. The animal is rewarded (generally, with food) for choosing the stimulus that matches the sample.
It turns out that, while a surprising range of species can learn this task equally well when the delay is very short (bees, pigeons, rats, sea lions, apes, dolphins, you name it), most species have remarkably short memory spans. Bees, those microscopic geniuses, can handle at most a few seconds’ delay, while in most birds memory span is in the range from 10 to 20 s. Mammals seem to do a bit better, a minute or so, but because data have been gathered from just a handful of species (we could find 25) we cannot be sure that this difference is reliable. Only pigeons have been extensively studied among birds, and it is perfectly possible that other birds species have memory spans comparable to mammals. What seems clear, however, is that humans can easily remember simple stimuli for much longer times (48 hours is documented, but it’s easy to imagine much longer memory spans, see the paper linked below for a detailed analysis of the data).
What have we learned from this review? We suspect that long memory spans are possible in non-human species only in the presence of specific adaptations for remembering specific kinds of information (e.g., food locations). Lacking such an adaptation, even simple stimuli like the colored lights often used in DMTS experiments are hard to remember, and there do not seem to be huge differences between species (at least, across vertebrates).
A preprint of the paper is available here.
Media coverage: National Geographic,
A few weeks ago I had the good news that our paper on the comparator model of associative learning had been accepted in Psychological Review. This is my first published paper co-authored with by an undergraduate student, Ismet Ibadullaiev, which makes me even happier. The paper (I put up an unofficial copy on my Papers page) deals with a very interesting model of associative learning in which most of the interesting phenomena are generated as memories are retrieved, rather than when memory are stored as assumed by most mainstream theories of associative learning (e.g., the Rescorla-Wagner model and its derivatives).
Our conclusion, unfortunately, is that the theory makes a number of paradoxical predictions that are hard to reconcile with empirical data on learning. For example, it predicts that, in many cases, animals would not distinguish which of two stimuli is most associated with a reward (they do distinguish, of course), or that they should learn equally about faint and intense stimuli (in reality, animals learn preferentially about intense rather than faint stimuli).
These problems have been hard to recognize because the theory had been studied exclusively by intuition and computer simulation. Both are fine tools, but they do run into trouble. The predictions of comparator, as it turns out, vary greatly depending on the value of a few parameters, and our intuition is not well equipped to reason about the non-linear effects that abound in the theory. Simulations give us correct results, but only for the parameter combinations we simulate. We have been fortunate enough to realize that one could write down a formal mathematical solution to the theory. With this solution it became much easier to see the big picture and actually prove what the theory can or cannot do.
I enjoyed working with comparator theory because of its distinct flavor – as hinted above, it’s rather different from other learning models – and because of the many surprises we had while exploring its predictions. Although we found what appear to be serious flaws in the theory, these might be more in its mathematical implementation than in its core concepts. The ideas that memory retrieval is an important factor in associative learning, and that stimulus-stimulus associations are more important than other models acknowledge, may well be worth pursuing. But the formulae that translate these ideas into a testable model will surely need to be revised.
A new paper of mine just came out in the Journal of Mathematical Psychology. It considers an old issue that has traditionally split the field of associative learning, and that echoes various scientific disputes between holism and reductionism. The question is, when an animal learns about a stimulus, how is the stimulus endowed with the power to cause a response? Configural models of learning assume that a mental representation of the stimulus “as a whole” acquires associative strength (learning psychologists’ term for a stimulus’ power to cause a response), while elemental theories assume that the stimulus is fragmented in a number of small representation elements (say, shape, color, size, and so on), each of which carries some associative strength.
Long story short, it turns out that there is practically no difference in these two approaches. They amount to different bookkeeping of associative strength without this having necessarily any observable consequence. In fact, the main result of the paper is that, given some mild assumptions, for every configural model there is an equivalent elemental model – one that makes exactly the same predictions about animal learning – and, vice-versa, every elemental model has an equivalent configural model.
Thus there is no “better way” to think about how stimuli acquire associative strength, something that I expect will surprise some learning scholars. What I have personally most enjoyed discovering while working on this topic is that learning psychologists, and specifically John M. Pearce in this 1987 paper, have re-invented the formalism of kernel machines, a workhorse of machine learning and computer science since the 1960s. In fact, my proof of the equivalence of configural and elemental models is itself a re-discovery, in a much simpler setting, of the “kernel trick” of machine learning (see the previous link, and thanks to an anonymous reviewer for pointing this out).
Intriguingly, this is not the first time learning psychologists independently develop concepts that had been introduced in machine learning. Another remarkable case is Donald Blough‘s 1975 re-invention of the least mean square filter (or delta rule), a kind of error-correction learning that had been developed in 1960 to build self-regulating electronic circuits, and that Blough developed as a model of animal learning. I resist from speculating too much on whether this means that there is only one way to be intelligent – be it for animals or machines.
“Can pigeons read?” is the question asked at the beginning of this old video, aimed at illustrating techniques to teach animals complex discriminations by rewarding them for correct choices but not for incorrect ones.
These techniques, developed around 1930, have been used in a study teaching baboons to recognize English words from non-words. Soberly entitled “Orthographic processing in baboons,” the study has been often headlined “Baboons can read,” even by the very journal who published it. My colleague Johan Lind was delighted to hear the news: “If they can read, then I can write to them and ask about animal intelligence.” Unfortunately, the only thing the baboons would be able to tell Johan is which combinations of letters are more likely to appear in English words, which is what they learned by receiving food anytime they correctly identified four-letter sequences as an English word or a non-word.
The study actually demonstrates that you do not need to know language to tell words from non-words. All languages have a statistical signature, whereby some combinations of sounds (and, therefore, letters) are common, and others are rare. Baboons are smart enough, and see well enough, to learn this. I would not be surprised if pigeons could do it too, given that they can, for example, discriminate paintings by different artists, presumably learning something about the artists’ “visual grammar.” Pigeons can also associate different written words with different actions, as the video above shows. All this suggests that the evolutionary origin of our ability to read is even more ancient than “reading” baboons suggest, pigeons being separated from humans by some 150 million years of independent evolution. Analyzing the structure of visual stimuli is a natural task for many animals, and I do not think the key to understanding human uniqueness lies here.
We have prepared a flyer to advertise the Conference on Human Cognitive Uniqueness that will take place at Brooklyn College on May 29-30. Feel free to use it to advertise the Conference yourself!
Do animals have emotions? This is tough philosophical question – how would we ever know? How do you even know I have emotions? Traditionally, we have recognized emotions by their outward manifestations alone. We see someone laugh and we assume they must be happy because we laugh when we are happy. In the case of animals, relying on such inferences means that it is easier for us to attribute emotions to animals that are similar to ourselves, such as apes and monkeys, than to animals such as chickens, who show no facial expressions and whose behavior can be interpreted reliably only by experts.
There are other ways to infer emotions, for sure. When we see someone offering help, we usually assume they care for others. A recent report shows that chickens care for others, too. Hens react with distress calls and accelerated heart rate to their chickens being distressed by puffs of air. Is mother hen feeling bad for the chicks? We cannot know, but we should not just assume they can or cannot based on what we see and how we like them. We need to understand what emotions are and what a brain needs to generate feelings and consciousness. Today, we simply don’t know.
We should not be surprised however, that mothers in species with parental care show outward signs of emotions. The mother is there to nurture offspring, and she must react to potentially threatening situations. The outward signs of emotions we can measure are simply these defense mechanisms: the mother’s call tells both the chicks and potential predators that she is there, and her increased heart rate is a sign that she is getting ready to defend the chicks.
If we were to take these behaviors at face value as signs of feelings such as empathy for others, we would conclude that ants and bees are more empathic than chickens, than apes, than humans. Ants and bees do no hesitate to get killed to defend their nests, for example, and they do so with unerring resolution unknown to us, apes, or chickens. These behaviors are probably just genetically programmed reactions and we do not routinely assume that ants and bees feel bad for the eggs and larvae that are in danger when the nest is attacked. Konrad Lorenz pointed out in 1935 that the situation is quite complex:
“The Jackdaw (Coloeus monedula) possesses a very interesting reaction of defending any fellow-member of the species in the grip of some bird or animal of prey. For a long time I have been familiar with the fact that my tame but free-living Jackdaws would furiously attack me if I gripped one of them in my hand, but I was very much astonished when I inadvertently elicited exactly the same response by carrying a wet, black bathing-suit in my hand. Subsequent experiments showed that anything glistening black and dangling, carried by any living creature would release the very same reaction in the Jackdaws. Even Jackdaws themselves were subject to attack from their fellows when they happened to carry nesting material possessing the characteristics just mentioned.”
To summarize: We cannot know what a hen, a bee, or any other organism feels because we do not understand how brains can produce feelings, and any appearance of feeling can be mimicked by the notorious mindless zombie of consciousness philosophy.
[Update: A slightly more technical piece on this topic has been published in PNAS]
BBC News report that, in recent experiments, rooks (a species of crow) have demonstrated surprisingly sophisticated tool use. For instance, the rooks learned to insert a stone into a plastic tube to gain access to a second stone, which they then inserted into another tube to finally retrieve a juicy maggot.
I am a big fan of corvids. But what do these new findings say about their intelligence? What do rooks understand about causes and effects in the physical world? The controversy that this experiment touches upon boils down to the question: How much did the rooks figure out on their own, and how much did the researchers help them? Time-honored animal training techniques, in fact, allow to "shape" (as animal psychologists say) behavior of amazing complexity. Just think of what animals do in movies. The key technique is to break down a complex behavior into small, simple components that the animal can learn without much difficulty (and without much understanding).
The rooks' tool use behavior was shaped at least to some extent. For instance, stones where initially placed near the rim of the plastic tube, so that they could easily (and accidentally) be nudged down the tube. After the rooks mastered this step, stones were placed besides the apparatus, and finally they were moved further so that rooks learned to pick them up and transport them for some distance to the tube.
Such a use of shaping does not exclude that, by the end of the experiment, the rooks had developed an understanding of the task. For instance, they reliably chose stones small enough to fit into the tube. Thus I am not criticizing this brilliant experiment. I am
an even bigger fan of corvids now. But we do not know whether the rooks could have understood everything on their own.
Indeed, we do not know what "understanding" means. Animal psychologists have traditionally contrasted "insight" and "trial and error" learning. Insight is what happens when you realize the solution to a problem in your head, using your knowledge of causes and effects in the world. It is considered an advanced cognitive skill, available to humans and perhaps apes and, now, corvids. Trial and error is a more mundane process, whereby an organism learns to repeat behavior which, performed randomly or accidentally at first, has brought about desirable consequences. Shaping exploits animals' abilities of trial and error learning, by rewarding them only for the behavior we want them to produce (a classical example here). Can we understand animal intelligence by contrasting insight and trial-and-error learning? We do not know. Perhaps they are not fundamentally distinct phenomena, and a deeper understanding will come from looking at the problem from a different angle.
Main reference: Insightful problem solving and creative tool modification by captive nontool-using rooks, by C. D. Bird & N. J. Emery.
drghirlanda 
