The ambition for all children to read music is "far from being realised" while schools struggle with funding, music associations have said.
Nick Gibb, schools minister, has said that he wants every child to leave primary school able to read music.
But Deborah Annetts, chief executive of the Incorporated Society of Musicians, said this would require addressing the teacher shortage.
"To ensure every child leaves primary school being able to think and act musically, reading music as part of this, more emphasis must be placed on the subject within the primary curriculum, and the significant skills gap and shortage of teachers addressed," she said.
"With falling teacher recruitment, lack of specialist knowledge and a squeeze on funding, Nick Gibb's ambition is far from being realised."
Mr Gibb announced earlier this month that a new model curriculum for music would be developed by a panel of experts with the aim of ensuring all pupils benefit from “knowledge rich and diverse lessons” and help make planning lessons easier for teachers.
Separately, in a piece in the Times, Mr Gibb said that the panel was set up because of his concern that too few pupils are benefiting from a “sufficiently rigorous” approach to music.
And he added: “I want every child to leave primary school able to read music, understanding sharps and flats, to have an understanding of the history of music, as well as having had the opportunity to sing and play a musical instrument.”
Bridget Whyte, chief executive of Music Mark, a music education association, who is one of the panel members creating the new model music curriculum has also said that teaching primary children to read music would require "immense" investment.
Ms Whyte said that Music Mark welcomed the opportunity to be involved in the development of the new curriculum.
She said: “Whilst the aspiration that all children leave primary school able to read and write music is interesting, the significant investment needed to realise that aspiration through training and resources is immense.
"We also need to be careful not to dismiss the value of other styles and forms of musical learning which develop musical literacy and curiosity which should provide a strong foundation to a life-time of musical learning, playing and enjoying beyond Key Stage 2.”
The existing national curriculum for music, which has to be followed by maintained schools, states that during key stage 2: “Pupils should be taught to sing and play musically with increasing confidence and control. They should develop an understanding of musical composition, organising and manipulating ideas within musical structures and reproducing sounds from aural memory.”
The curriculum, which was introduced in 2014, includes six specific skills that children should be taught including: “use and understand staff and other musical notations”.
A recent survey by Sussex University found more than a third (35.8 per cent) of secondary schools had fewer members of staff in music departments than two years ago.