He trails in and out, answers questions elliptically, bickers with Griselda. Maybe he really was bullied by his agent into doing this rare interview and is starting to regret it.
Then, halfway through lunch, he warms up. We're talking about his novel, The Owl Service, which features a dinner set patterned in such a way that the heroine can trace either owls (bad) or flowers (good) from the markings. Garner's arm shoots out, gesturing towards a china plate hanging on the kitchen wall, a family heirloom. And there they are - delicate owls perched menacingly around the edge. "Griselda actually did it - she traced an owl on to paper and bent the paper so that it sat on the back of a chair."
The tension dissipates. Once we're in his study, he's charm itself.
Garner, 65, is one of Britain's best known children's authors, despite his grouchiness at being pigeonholed as a children's writer and the fact that The Stone Book, his last significant book for young readers, was published more than 20 years ago. There have been other, shorter, tales since.
In several of his handful of novels - "not prolific", he mutters - edgy youngsters are engulfed in the unfinished business of other worlds as forces from British myth and history burst into the late 20th century.
The Weirdstone of Brisinghamen, The Moon of Gomrath, Elidor, The Owl Service and Red Shift are classics - set texts on school reading lists and exam syllabuses; the subjects, even, of doctoral theses. They are also terrifying - they give my nine-year-old nightmares but nothing can stop her re-reading them.
Will he write another children's novel? Well, Grey Wolf, Prince Jack and the Firebird is coming next year from Scholastic in a series of folk tales re-told by well-known authors. But that's not the real work in hand. On the table in his study lies a grey box-file labelled "Thursbitch" - Old English, he explains, for The Valley of the Demon. In 1345, he adds, "they called things by what they were". It's Garner's next book.
Picking up the file, he lays out a series of photos and we're off on a rollercoaster account of how a Garner novel comes to be written.
It's an obsessive, painstaking process, a harnessing of mental powers almost beyond control. In 1989 Garner was diagnosed manic depressive. Now his pre-lunch grumpiness is explained - life is worst in the morning. But the illness's lurid highs and lows can be creative - sufferers have an ability to make connections where others see nothing.
Garner has learned mental tricks to control the beast, although not in time to prevent a serious collapse. During the filming for TV of The Owl Service in 1969, he had to be restrained from assaulting an actor.
In 1980 came another incapacitating attack. None the less, he says he is not "a severe manic depressive. Most of the time I can use my highs constructively, but there are times when I know the moon is going to hit the earth and I become highly distressed.
"Killing yourself," he adds quietly, "is not a nice thing to do to your family - you cannot help the people who are left behind."
The new novel is set, like most of the others, in Thursbitch valley, in Cheshire's Alderley Edge, 10 miles from Garner's home. His fiction is rooted in the strangeness of the Edge, its past, its people, their dialect.
At 22 he gave up on an Oxford classics degree to write in the house he grew up in and where, as a sickly only child, he sketched out imaginary worlds on the bedroom ceiling.
The photographs he pulls from the file are of standing stones in the Thursbitch valley, dating back to the Bronze Age. One stone streaked with white stands out. It's the centre, he thinks, of a circle used by the Celts thousands of years ago for stellar and lunar observations.
Then there is his discovery in the same valley of a memorial stone for a packman who was cast adrift in a snowstorm and died just half a mile from home.
Macclesfield, the great silk capital, is a few miles to the east. The question powering Garner's research is: "What if, at the time silk arrived in Macclesfield in the 17th and 18th centuries, it brought with it all the stories of the Silk Road and found something working in the hills?" More he will not divulge. "If I tell before something is in print, it's like fairies' gold turning into leaves in my pocket."
And so he will go on, scrupulously following his threads until he can see connections in the "chaotic mess" of his raw material which he began to tackle in earnest three years ago. Is this a children's novel, then, this book which scared him so much when he started it in 1952 that he abandoned it? "I do not distinguish," he says.
Garner has filled filing cabinets with readers' letters, many from children and teachers - and still they come, though today it is a trickle rather than a spate. Some are wonderful expressions of how the books gripped their young readers. Others, from schools, are, he says, "awesome in the horror they induce in me", full of "plonking" remarks.
He is ambivalent about being a "set text". He does not think English literature should be taught at all in schools, because it shackles children's "headlong enthusiasm" for reading.
Garner has a heroic view of teachers - there is, he says, no higher calling. He is passionate in his praise for the teachers at Manchester Grammar who opened the door to Oxford for him (his son Joseph attended the same school). And he is very helpful to Cheshire teachers, visiting classrooms, inspiring them with stories and wonder about the history and landscape of which they are part.
His most recent commitment is his role in a two-year community project to trace Alderley Edge's history and landscape. Manchester Museum, the University of Manchester and local schools are all involved, with schoolchildren taking part in workshops to explore how Bronze Age weapons were made and in a literacy project organised by Griselda Garner which includes the study of some of her husband's novels.
Yet Garner's regard for excellent teachers is overshadowed by his contempt for the damage he believes has been caused by progressive teaching methods. A generation has grown up, and is now teaching, which doesn't know the right questions to ask about his novels. They fail to notice their linguistic subtlety, their layers of research, the allusions to Old English, to the Classics, to history and archaeology.
On this, as on so much else, he is bluntly eloquent. He is in favour of selective education, of excellence, of teaching the Classics, of able children being stretched intellectually. He is against standards being lowered to allow more working-class students to go to Oxbridge.
These are unfashionable, alienating views. You understand why, at one teachers' conference, he was called elitist. And why he says: "I must come across as an old grouch, full of prejudice."
They are also the views of the man who has created some of our most haunting books for children. Does this make them worth listening to?