In its first report, the Advisory Committee on Mathematics Education said a radical rethink was needed to stop pupils leaving school without basic maths skills.
It says a national academy would revitalise maths education by raising the status of teachers and improving teaching quality.
The academy would link with new local maths centres where teachers and lecturers would work together on continuing professional development.
The committee was set up this year by the Joint Mathematical Council of the UK and the Royal Society, supported by the Gatsby Charitable Foundation.
Sir Christopher Llewellyn-Smith, ACME chair said: "The under-supply of numerate graduates means it is difficult to recruit new teachers with good quality mathematical backgrounds."