Gerald Haigh investigates how the brain makes sense of what we see
Look at your eyes in a mirror. In the centre of each is a black hole - the pupil, just a few millimetres across. Now consider that everything you see as you work, drive, enjoy the view at the seaside, read a book, gaze on your children - all of your world in fact - goes in through those two little holes to be projected as an image, upside down, the size of a postage stamp, on the retina at the back of the eye.
So far this is probably familiar territory - the structure and function of the eye, revealed in a cross-sectional diagram showing the cornea, the lens, the retina and the optic nerve. What we don't often consider - and what is surely the real miracle of sight - is the process by which the brain takes that small flickering upside-down image and uses it to construct the full majesty of stereoscopic, moving three-dimensional human vision.
Are we born with that ability? Many living organisms, after all, act on visual stimuli from the moment of birth - nestling birds hide from a flying predator, even a simple schematic model of one. Does a newborn human have anything like that, or is a baby faced with an incoherent jumble that has to be sorted out over time by the growing and developing visual system?
It's obviously difficult to carry out controlled experiments on human babies, so it's not an easy question to answer. However, the general conclusion is that, no it's not just a jumble, but, yes, there is an awful lot of learning to be done - part of being human, it seems, is having fewer innate skills and more that are constructed by the brain.
The "not a jumble" position is arrived at through the work of experimental psychologists who have discovered that babies as young as two days can distinguish between different printed patterns, and prefer those with curves. They can also - again effectively at birth - tell the difference between a cartoon-ish picture of an oval face (hairline, eyes, eyebrows, nose, mouth) and one with the same facial features arranged randomly on the basic oval. (This methodology was developed by Robert Fantz, and described in 1961 in Scientific American; it consists of measuring how long the babies choose to look at the different patterns.) A new-born baby hasn't had time to learn this, and so it's innate - an inherited characteristic and part of the story of human evolution.
That's just the start, though. Upon that innate framework, the full extent of which is still not known, is built an elaborate structure of learned visual skills and insights culminating in the adult's ability to build a fully functioning mental construct of what's out there. We take this constructional ability for granted, because we live with it, but it's astonishing stuff, and has fascinated philosophers and scientists for centuries. The great 19th-century pioneer of the psychology of visual perception, Hermann von Helmholtz, put it very clearly: "Visual perception is intelligent decision-making based on limited sensory evidence."
In discussing this, psychologists think of two sorts of processing - "bottom up", the straightforward acceptance of what's in the retinal image; and "top down", which means using what we already know, or have learned, to fill in or complete the image. Clearly, we need both. "Top down on its own is dreaming," is how visual psychologist Richard Gregory puts it. Professor Gregory, for many years a leading figure in this field (his highly accessible book Eye and Brain is in its fifth edition) sees "top down" processing in terms of the brain's continual drive to construct objects from the patterns on the retinal image. "It goes back to the gestalt psychology of the Twenties," he says. "Art depends on it. A cartoonist makes what looks like a few doodles and you see objects in it."
The brain, he says, sets up a hypothesis as to what it's seeing, based on the "bottom up" information, and then completes the picture "top down" from what you know already. So, for example, a few lines and curves on the retinal image lead you to decide that what you're looking at is a cup of tea. You fill in the rest, and your behaviour - reaching out, looking for a handle, being careful in case it's hot, trying not to spill it - are determined by your hypothesis and not by the direct stimuli.
That this is how it works is confirmed by the relative ease with which we can make mistakes - so we see the Loch Ness monster in a pattern of shadows, or the face of the Virgin Mary in a cheese toastie. In each case the brain has found something in the pattern presented by the retinal image that corresponded with its learned or inherited experience of objects. As Gregory writes in Eye and Brain: "Any retinal image is infinitely ambiguous, though generally but one interpretation is seen."
In terms of the survival of our species, it's the kind of process that enables you to spot a lurking enemy, or even to find a mate. So it gets you into trouble once in a while? That's life, my friend.
Perception has many layers, many links, many subtleties. It's possible to cut to the chase, though, by thinking of three main perceptual abilities - depth (or distance), movement and colour. Each is vital, each is dealt with largely by a different part of the brain (though there's increasing understanding of their interdependence) and each is capable of being impaired - which is important, because it's brain impairment and injury that so often provides clues to what's going on when everything works.
Perception of depth and distance
The retinal image is flat, so how do we "read" depth and distance into it? Part, but by no means all, of the answer lies in the fact that we have two forward-facing eyes. These swap the all-round vision of, say, rabbits, or many birds, which need to be able to see everywhere at once if they're going to survive, for overlapping fields of vision. The overlap is good for hunters such as hawks, tigers and human beings, because it gives stereoscopic vision. Hold a pencil just ahead of your nose and look at it first with one eye closed and then the other. You get two different views of it, which the brain fuses together to produce depth.
That sounds easy enough, but many years of study and observation have been devoted to some of the questions that arise. How, for example, does the brain know which bits of the two retinal images belong together to make a stereo pair? It can't be just knowledge of which object is which (that's a pencil, and, yes, that's a pencil too), because it works regardless of that. So there are subtle rules at work which are incompletely understood.
But judging distance is more than just stereoscopy, which operates only at relatively short range. A person with one working eye can still perceive distance, through "occlusion" (distant parts of the scene being hidden by nearer parts) and clues such as colour and shading, all of which depend on prior knowledge. Richard Gregory recalls beating the eminent biologist Maynard Smith at tennis: "He only had one eye, so I'd lob the ball high, and against the sky, with no background, he couldn't judge it," he says, grinning at the recollection.
Perception of movement
Movement is relative - something moves by comparison with something else.
The brain has to decide, every time, what's moving and what isn't. The default position it adopts is that small objects move and big ones stand still. It's a reasonable starting point, but it can be wrong, which is why the moon (small) seems to race past the clouds (big).
Then there are the movies, which work because we see a rapid succession of still pictures as a moving scene. This is typically explained by "persistence of vision" - the fact that an image stays on the retina after the stimulus has gone, as Richard Gregory says, though there's more to it than that.
"There are two things going on. Persistence of vision prevents you from seeing flicker, but there's also the brain's ability to see movement across gaps." (The same mechanism is at work if you flash two separated lights at the right speed against a dark background - you see one light moving to and fro.) The brain's involvement is confirmed in a case described in medical literature, of a stroke victim who found herself living in a static world.
Her vision was fine, apart from seeing movement. Water pouring from a kettle looked like a frozen column and people walking about would disappear and then reappear in a new position. She learned to cope, largely because she remembered what normal vision was like, and was able to make coping assumptions - that a car on the road, though apparently still, was moving towards her, for example.
Perception of colour
At the back of the eye is the retina, the screen on which the images we see are focused. In effect the retina is an outgrowth of the brain that has evolved to be sensitive to light. "Receptors" on the retina pick up light and pass it on, first to the optic nerve system and then to the brain.
There are two kinds of receptor - rods and cones. The cones - there are about seven million of them - work in daylight and give colour vision. The 120 million or so rods work in low light, but only give shades of grey; they are particularly sensitive to motion at the periphery of the visual field. The cones in the normal human eye are sensitive to three colours - red, green and violet - from which the brain builds the complete visual spectrum. But it's not just a matter of physics, for the brain does its own thing with the apparently objective colours that we see. This is a whole field of study in itself, but there are some simple, observable examples - an orange becomes noticeably more orange in colour when we realise what it is. A candle flame that is yellow in daylight appears to be white in a dark room.
Recovery from blindness
Vision scientists are deeply interested in people who have recovered their sight, because they think it should be possible to gather from them, by experiment and observation, useful information about how the brain learns to build on visual input. Such cases are extremely rare - there have been about 20 in the past 200 years, and most of those were complicated by uncertainty about exactly when sight was lost, and also by the imperfections of early surgery.
The most reliable evidence comes from two relatively recent cases. The first, studied by Richard Gregory and Jean Wallace, is that of Sidney Bradford, who received successful corneal transplants in 1959 when he was 52. We imagine that someone in Sidney's situation would be out and about, grinning with joy and wonder. In fact, it isn't like that at all. A blind person inhabits a world without vision, and the sudden shift to a different world is disconcerting, confusing and often disappointing. Some things - those you know by touch - emerge from the confusion quickly. Others come more slowly, and there are tensions, mistakes and embarrassments along the way that are enough to challenge the sternest of emotional constitutions.
So it was for Sidney Bradford. From the start, he recognised things he knew well by touch - a clock face, because he had a special watch that he could feel, capital letters, because he'd learned at school to recognise raised letters by feel. Less immediate and tactile experiences were confusing - he couldn't judge distance well, thinking that he might touch the ground from a high window. As time went on, he became despondent. He lost confidence, found the world drab and the traffic frightening. He would often sit in the dark at home, preferring the comfort of the familiar. He had, Gregory concluded, lost more than he had gained.
The paper on Sidney Bradford, then known as "S B" (Experimental Psychology Society Monograph No 2, 1963, RL Gregory and JG Wallace) reported: "Before the operation he was regarded by everybody as a cheerful rather dominant person, and we independently formed this opinion when we first saw him at the hospital. He seemed changed when he came to London; dispirited and bored. It seemed to all of us that he was deeply disturbed; yet too proud to admit or discuss."
Sidney Bradford's decline continued, and he died on August 2, 1960. Gregory and Wallace conclude: "His story is in some ways tragic. He suffered one of the greatest handicaps, and yet he lived with energy and enthusiasm. When his handicap was apparently swept away, as by a miracle, he lost his peace and his self-respect."
Mike May, in the US, would understand all of Sidney's feelings. Mike had a corneal graft in March 2000, at age 46, having been blind since he was three. He, too, was a confident outgoing blind person, a sportsman who had married his ski instructor.
Crucially, he is a wise and intelligent man, supported by a wife with deep understanding of all that's happened to him. Consequently, he's worked his way through the experience of acquiring vision with the aid of humour and a questioning nature - on his first flight as a sighted person he asked his seat partner, a stranger, to explain the confusing patterns of sky and ground and sea that slid past his window. Now, it seems, Mike has found a place, somewhere between seeing and blindness, from which he dips into the skills of one or the other as the need arises. It's a remarkable story, accessible in his diary at www.senderogroup.commikejournal.htm
One excerpt, written about three weeks after his operation, illustrates the difficulty of learning to integrate vision with other senses, the possibility of frustration that in some might lead to real disappointment, and, above all, Mike's ability to articulate his difficulties and handle them with good humour and resilience: "I found it very distracting to look at people's faces when I was having a conversation. I can see their lips moving, eyelashes flickering, head nodding and hands gesturing. First, I tried looking down and if it was a woman, a low-cut top would be even more distracting. It was easiest to close my eyes or tune out the visual input.
This was necessary often in order to pay attention to what they were saying. I am sure there will come a time when all this visual communication will mean more to me but for now it is just distracting."
In the context of the restoration of sight to the blind - a biblical miracle - the phrase "just distracting" is surely very telling.
There's much to learn from people like Sidney Bradford and Mike May.
However, according to Richard Gregory, we have to be very careful before assuming how far their experiences are related to the development of vision in a young child. Children go through a long period of making sense of the visual world, helped by direct experiences of touch. But they do not have the years of experience of "touch" and imagination of a blind adult, nor do they carry the same emotional burden.
What they do possess is a built-in drive to make sense of the ever-increasing detail that the developing visual system delivers. So, for example, work at Cornell University Baby Lab shows that, within four months, a baby understands "trajectory" - that an object travelling along a path is there all the time even if only appears fleetingly. This understanding becomes hard-wired and underpins a mass of further visual learning. So, to compare the alert and excited face of a baby with the downcast demeanour of Sidney Bradford is to realise that what was sad and depressing for Sidney, is intriguing and full of promise for a new-born infant.
Cornell University Baby Lab http:babylab.psych.cornell.edu