An HMIE report into inclusion warns that the levels of achievement of some young people, particularly those looked after by local authorities, remain "unacceptably low".
In some cases, the root cause is uninspiring teaching and low expectations from teachers, it adds.
Although the Scottish education system has "much to be proud of" in its inclusive approach, there remain "substantial inequalities in outcomes for children, young people and adult learners", says senior chief inspector Bill Maxwell.
To address this, teachers need support in initial teacher education and continuing professional development, says the report.
Count Us In: Success for All, published last week, says there is "insufficient day-to-day challenge" for some children and young people, including those who are capable of very high achievement in particular areas.
Instead, there is a "narrow, slow-paced approach which stifles creativity and leads to loss of interest"; in the worst cases, it leads to frustration and depression.
The report continues: "Some children and young people, such as young carers or those with challenging family circumstances, have difficulty in meeting day-to-day expectations for punctuality and homework.
"Too often, staff respond by reducing their own expectations of what these learners are capable of, sometimes in the belief that they will encourage the learners by helping them avoid failure. However, such children and young people have high levels of responsibility outwith school, which often go unrecognised."
Many learners find some teaching approaches uninteresting and respond by disengaging from learning, say the inspectors. In turn, staff assume a lack of ability and lower their expectations.
The inspectorate's report comes at a time when the numbers of children and young people with additional support needs, including complex needs, has steadily increased over the years.
"It is no longer only the responsibility of specialist staff to support the wide range of learning needs of children and young people. This support is now regarded as being everyone's job," adds Dr Maxwell.
The report is also intended to be a good practice guide, with illustrative film clips linked to The Journey to Excellence resource. It includes a series of reflective questions about how current practice can be improved.
Elizabeth Buie email@example.com. www.hmie.gov.ukpublications.aspx.