Graham Donaldson, senior chief inspector, told a conference on good practice in inclusion at Heriot-Watt University this week: "We have a long way to go to get all schools up to the high standards that only a few currently achieve."
Across the sectors, only one school in six has major strengths in appropriate learning and teaching, Mr Donaldson said.
Evidence from inspections over the past three years shows that a quarter of all schools have been graded as only fair or unsatisfactory on their provision for individual pupils.
In contrast, Mr Donaldson said virtually all schools (97 per cent) were good in their overall approach to promoting fairness and equality and half of them were very good. Similarly, 87 per cent of schools had developed an ethos of achievement.
Pupils surveyed confirmed that the vast majority enjoy school and are positive about their experiences. Some are more critical as they move into secondary.
Mr Donaldson reminded his audience of Scotland's high standing in international league tables, especially in maths and science, but appealed to schools and local authorities to work harder on closing the gap between the highest and lowest performers.
Since 1999, the percentage of pupils with five or more Credit awards at Standard grade had risen from 37 per cent to 47 per cent but the performance of the lowest achievers in S4 stayed flat. "The pace of improvement for the lower end of the range is not keeping up," Mr Donaldson said.
Integrated community schools were tackling inclusion on a broader front, although HMI's evidence showed that the impact of the initiative was "limited and confined to relatively isolated pockets of good practice".
Mr Donaldson summed up: "We are still well short of meeting the needs of all young people. The opportunity gap remains too wide. Schools can bridge that gap by high-quality teaching, an imaginative curriculum and a better focus on individual needs."
The education and children's agenda had to be brought together more closely. "Inclusion is not easy but it's also not optional," Mr Donaldson said.