Young people in the Neet (not in education, employment or training) group often look back with fondness on a single adult in an otherwise unhappy school career.
Many youngsters lack role models, with no one at home either holding down a job or having completed secondary education.
These are some of the findings of the Engaging Youth Inquiry, a joint project from the Nuffield Review into 14-19 education and Rathbone, the educational charity. It was based on focus groups with young people and youth workers in Manchester and is the latest part of a major inquiry into Neets.
Dr Stephanie Wilde, of Oxford University and one of the report's authors, said: "One of the things that came up quite commonly in the workshops was that the young people remembered one teacher who had not shouted at them, and treated them like an adult. A lot of them were quite nostalgic about individual teachers."
One 16-year-old boy said: "One teacher was different, knew what to say. He taught maths. He was just normal and down to earth."
Another 16-year-old boy said: "One (teacher) was safe, didn't shout at you if you did something wrong ... took you out of the room to talk to you instead of making you look stupid. There's always one, though, isn't there?"
But as these anecdotes suggest, the young people's interactions with adults were largely negative. An 18-year-old boy said: "I didn't really like it, didn't get on with any of the teachers - all stuck up."
Others appear to have rejected school from the age of 11.
Many 16 to 18-year-old Neets suffer from "an ingrained sense of failure", often beginning in primary school, the report said.
Also, the Government's emphasis on GCSE A* to C grades as benchmarks of success has "serious implications" for lower achievers.
Professionals working in the field said that young people needed to value themselves before they could value education, and that having a trusted adult was crucial to this.
One youth worker said: "Many do not have people to inspire them, apart from Wayne Rooney, media stars or people on the estate who make loads of money by dealing.
"They have to feel that the system has a place for them."
The report also includes insights into inner-city neighbourhoods. A 17-year-old described his area as full of "smackheads, gunshots, dirty shops, drug users". Asked how his area could be improved, another said: "Stop the 10-year-olds being out at night terrorising everyone."
The report said it was crucial to give adults space to build trusting relationships with troubled teens, based on listening and negotiation.
The term Neet itself was counter-productive, the report said, because it was not descriptive enough of the wide range of individuals involved. The final report is due in October.