"There are still too many schools where expectations are unacceptably low, which let down the pupils who attend them," they said.
The councils are being asked to outline the steps they will take to improve attainment generally and improve schools that have been failed by Ofsted or missed minimum targets. They will be expected to discuss their solutions with Sir Bruce Liddington, the new schools commissioner, and with national strategy officials.
Lord Adonis said: "It is now time to set some fresh challenges ahead and to build on progress in partnership with local authorities.This year's provisional national results show a mixed picture. Just one pupil let down is one too many."
Ministers are particularly concerned about the slow rate of progress in raising primary standards. The letter welcomes the improvements that have been made, but said: "There was no progress in the proportion of 11-year-olds reaching level 4 plus in English and only a small improvement in mathematics. Many pupils who should be progressing two levels across key stage 2 are not doing so."
Ministers are also disappointed about the fall in key stage 3 English results.
John Freeman, director of children's services in Dudley and vice-president of the Confederation of Children's Services Managers, said: "It is hard not to read this letter as negative.
"The issues they have identified we already know about and are already taking action on with the national strategies."