Schools, especially in Glasgow, are regularly slated for failing to match the results of others in neighbouring authorities, but Dr Armstrong fingers poverty and social exclusion as the root causes of educational disadvantage.
Recent studies, he states, have shown clearly that "higher childhood IQ might be related to better general health, the subsequent development of healthier behaviours and the potential to obtain safer and better-paid jobs".
He adds: "A growing list of factors related to socio-economic disadvantage and low social class are now known to affect the development of a child's mental ability and physical and mental health.
"These include maternal smoking, drinking, illicit drug use and poor nutrition during pregnancy, insufficient breast-feeding and lack of intellectual stimulation during the early years. Differences in trends of these behaviours among the social classes may at least partly explain the widening gap between the most and least affluent."
Dr Armstrong believes Scotland's health is improving but is not keeping pace with many of its neighbours. A "Scottish effect" is evident and that may be down to widening inequalities.
The chief medical officer argues that structural changes alone, such as a focus on economic regeneration, will not achieve the aim of countering poverty. Efforts must be made to increase the resilience and sense of control of people living in deprived areas.
Dr Armstrong notes that the average life expectancy of men has increased by 3 per cent over the 10 years to 2001, but mostly among the most affluent.
"Despite improvements, the less favoured sections of the Scottish population are falling behind," he reports.
Noting the current controversy about diet and school meals, he says that the food industry is "responding well to the challenge". He backs the Hungry for Success campaign to transform school cuisine.
Despite the upbeat report on healthy eating, he reminds the First Minister that one in three children is now overweight by the age of 12 and obesity among adults has tripled. Obesity rates continue to rise in children from all social backgrounds.
"The fundamental cause of obesity is the consumption of excess calories compared to calories expended - in short, physical inactivity and poor diet. The result is excess body fat, which frequently leads to considerable health impairment," Dr Armstrong states.
Overall, children in poorer areas continue to suffer disproportionally.
Hospital admissions for asthma are 25 per cent higher, they have have the highest rates for unintentional injuries and more are admitted with diabetes - cases rose by 20 per cent over the 10 years to 2003.
While there has been notable success in reducing teenage pregnancies - down by 12 per cent in the eight years to 2003 - rates in the poorest areas are up to three times higher.