But, though Mrs O'Keeffe, a nurse, was sure there was something wrong, getting doctors to agree was another matter: "They would say that it would pass, or accuse me of trying to label him." Finally, "by constantly fighting", she got a diagnosis of autism. Recently that has been refined to Asperger Syndrome.
Even though Connor received a statement which assured him a full-time support assistant, four or five local schools refused to take him. "As soon as they knew he had special needs they didn't want to know," says Mrs O'Keeffe. Some suggested he would be a bad influence on the other children, others didn't think they could cope.
Then two years ago, he was accepted at Gastrells community primary school in Stroud, Gloucestershire. The school was "totally welcoming". The improvement in his attitude and behaviour, Mrs O'Keeffe says, has been dramatic; she is delighted with his progress.
"Two years ago he asked me, 'What is a friend?' He had no concept of friend. Now he says he doesn't have friends, 'but I do love the other children'."
It took 18 months to get him to put his hand up, but he does, and he has begun to understand the needs of others. Moreover, Connor is now able to manage for one afternoon, plus half an hour at lunchtime without his support assistant.
Academically too "he is holding his own". His maths is poor, but his reading is good, and he has islands of what you might call extreme ability. For example, a precocious understanding of photosynthesis and physiology.
Gastrells School includes a centre for 10 children with receptive language impairment. Connor doesn't attend this, but it does mean that the school's teachers have a lot of experience of children with special needs.
Moreover, "pupils are used to interacting with children who might be considered different", says his headteacher, Cottia Howard.
When he began school his behaviour was certainly different. He was, for example, still in nappies; he would sometimes sing and make loud noises. He was very scared of the hall, with its wide spaces and lots of other children and would scream if asked to approach it. Gradually, however, he has mellowed. "At first he had to be told exactly what was going to happen during the day at every turn, and informed of changes, but he is now far less reliant on this," says his teacher Lorraine Rich.
Connor's support assistant, Christine Smith, interprets and explains things to him and occasionally withdraws him to a quieter place; today this is usually not because of his behaviour but "because a particular level is not appropriate, or because he is finding it hard to focus," she says.
"He gets a great deal from one-on-one teaching," she adds. "But there has to be a balance - some classroom integration and some one-to-one."
Mrs Smith is also conscious that sometimes her role is to stand back, either to let Connor integrate on his own or to let other children step in to help; or to allow him to be alone.
No-one is saying that Connor could do without a large amount of support, but everyone at the school agrees that he is a joy to teach. "An asset to the school," is how the headteacher describes him.