Schools in Wales still have a "substantial journey" to make before they meet the key challenges set out by Estyn, according to its chief inspector Bill Maxwell.
In his annual report released this week, Dr Maxwell said there was some evidence of "positive progress", but that too many learners are still underachieving.
Although "encouraged" by some of the good practice he has seen in the past 12 months, Dr Maxwell said there was still much work to be done.
"There is a picture of gradual progress across the system, but with much scope for further improvement and the need for improvement to be made more quickly if we are to compete effectively," he said.
While most of the schools and colleges that were inspected in 2008-09 are performing well, the report says there is still too much variability.
More needs to be done to "tackle mediocrity" and bring the performance of the weakest providers up to the level of the best.
Generally, in both primary and secondary sectors, inspectors found better partnership working and improvements in the quality of leadership and management.
Standards are mostly good and improving, but the proportion of teaching judged to have major shortcomings remains "stubbornly static", the annual report says.
Self-evaluation, which plays a major part in the new inspection system being rolled out from September, is still weak in about a fifth of primary schools and a quarter of secondaries.
Local authorities, schools and governors must "raise their game" to address the issue, the report warns.
Dr Maxwell also told TES Cymru that he wanted local authorities to get better at making difficult decisions over surplus places and urged them to look at their whole school estate when considering reorganisation, not just one particular section.
"I want every pound to be spent improving teaching and learning in the classroom," he said.
"The Assembly government should work closely with local authorities to help them make the difficult decisions and to do so in a coherent way. We need to convince parents that they will get a better deal out of the new arrangements."
Teaching unions generally welcomed the report but expressed concern that poorer pupils are still underachieving.
Anna Brychan, director of the National Association of Head Teachers Cymru, said the report made "uncomfortable reading".
"Given the divergence from performance levels in England, it would be a brave person who continued to argue that the pound;500 per pupil funding gap doesn't have a part to play here," she said.
Dr Philip Dixon, director of the Association of Teachers and Lecturers Cymru, said: "We have been warning for some time that both ends of the cohort are not receiving enough help to achieve their full potential.
"We must be careful, as the chief inspector implies, that we don't become a country that is content with mediocrity. Some (local authorities) have made great strides in engaging children from deprived backgrounds, but others have not."
Disadvantaged pupils go short on support
Pupils from disadvantaged backgrounds are still not receiving enough support to reach their full potential.
Estyn's report says most schools are failing to monitor the progress of pupils from poorer backgrounds, with those entitled to free school meals still performing much worse than their peers in exams.
Dr Maxwell said it is not clear whether extra funding from the Education Maintenance Allowance or the Assembly's RAISE programme is having any impact.
Some RAISE funding has been misspent on pupils who are poor in educational terms rather than socio-economically, his report claims.
Dr Maxwell told TES Cymru that the situation could be improved by learning from the examples of successful schools.
In disadvantaged areas, the best schools have high expectations of standards and behaviour, and pay substantial attention to developing their pupils' social skills, confidence and self-esteem, he said.
Singled out for higher rates of progress
For the first time, the chief inspector's report includes several case studies on recently inspected schools that are making marked progress.
One of them, Pontypridd High, was the venue for the report's launch. The school achieved seven grade ones (outstanding) in its inspection, despite having an above-average percentage of pupils receiving free school meals.
Inspectors said Pontypridd High had developed strategies that successfully tackle social disadvantage and stereotyping.
Dr Maxwell said: "I'm very keen to develop Estyn's role as an inspectorate in promoting the best innovative practice. It can be very constructive to make that available to others. I don't think it's divisive; many providers will be featured in future."