An Office for Standards in Education statement said this week that the subject is in decline and "has become neglected and marginalised" in primary schools.
"Inspection evidence reveals that geography is the worst taught subject in the primary school curriculum. The rate of overall improvement in all aspects of geography, including teaching and learning, is slower than that of all other subjects," it says.
In secondary schools, pupils are turned off by a curriculum crammed with content rather than designed to stimulate their interest in the world around them.
Many geography departments have poured all their efforts into their GCSE and A-level classes, allowing their key stage 3 curriculum to stagnate. As a result, the number taking the subject at GCSE has fallen by a third in the past eight years.
David Bell, chief inspector, said: "Water shortages, famine, migrations of people, disputes over oil, globalisation and debt are all major issues with which our world is grappling and this is the geography of today.
"However, we have found that there has been a relative decline in the subject in recent years, with the picture particularly stark in primary schools.
"We need to engage pupils more purposefully and make them realise the relevance and value of the subject and ensure that they enjoy it."
Inspectors said the lack of specific teacher-training and professional development opportunities for primary teachers means many do not have a proper grasp of the subject.
As a result they are spending too little time on its study. Many primaries also lack the resources to teach it effectively.
In secondaries, pupils get little fieldwork experience. But there is one silver lining: inspectors praised a new geography GCSE course, being piloted by the OCR exam board.
A Department for Education and Skills spokesman said the Government is determined to revive the subject and will appoint a chief adviser for it in March.
Dr David Lambert, Geography Association chief executive, said the Government is providing pound;100,000 next year to develop the subject in primary schools.
He said: "I am very pleased that the Government has this year started to take this issue seriously but more needs to be done."