Mr Cameron told a London conference that a dearth of post-16 special needs provision meant that many pupils were forced to choose between college or staying at home.
The Tory leader also highlighted new findings, which show that special schools are more likely to be closed down than mainstream schools, despite performing better.
While special schools were six times less likely to be judged "inadequate"
by Ofsted, they were three times more likely to be shut.
Lord Adonis, schools minister, has announced councils need to justify future closures, but Mr Cameron dismissed the move as "sticking plaster on a large wound".
More controversially, Tories are considering whether poor teaching and class organisation are to blame for some special needs diagnoses.
"Many children are being diagnosed who needn't be," said Katie Ivens, a member of the Tories' special needs commission. Moving pupils to face the front and teaching with synthetic phonics could be helpful in milder cases, she said.
Adam Willis, an educational psychologist specialising in autism, said there was some truth in this.