The traditional primary school class teacher could be holding pupils back, according to the Office for Standards in Education.
OFSTED is urging schools to use more subject specialists - but at the same time it recognises that specialist teaching will mean employing extra staff.
A new report, Using subject specialists to promote high standards at key stage 2, says that the standard "one teacher, one class" structure places a heavy burden on the class teacher. It also prevents subject experts from spreading their knowledge.
"Although there are some notable exceptions, we are left with a disappointing picture in which there is an under-use of talent," says the report.
"For example, PE specialists limited in their contribution beyond their own class to some after-school games clubs; the theology graduate teaching religious education only to her class, despite a general lack of confidence elsewhere."
Teachers with expertise, say the inspectors, find it hard to influence lessons in the rest of the school.
The report says that schools should abandon the traditional structure and allow specialists to teach classes other than their own. According to OFSTED, a number have already raised their teaching standards in this way.
However, the report acknowledges that subject teaching presents difficulties. Even the most successful schools, it says, encounter staffing problems in terms of either a lack of non-contact time or a lack of expertise, Primary schools will need more staff in order to make more use of specialist teaching without damaging the class teachers' pastoral work.
"Stated bluntly, schools need more teachers than classes if they are to manage subject teaching effectively," the inspectors conclude.
"Using subject specialists to promote high standards at key stage 2: an illustrative survey" can be obtained from OFSTED Publications Centre, PO Box 6927, London E3 3NZ. Tel: 0171 510 0180.