Baroness Kennedy urged Frank Field, the minister in charge of welfare reform, to introduce a "learning allowance" to ease the path back into education. Government departments had to work together.
"One of the most terrible inhibitors of participation is 'if I go to a class, I'll lose my benefit'. They feel they will lose the raft on which they float," said Baroness Kennedy, who chaired a recent committee on widening access for the Further Education Funding Council south of the border.
She warned that many politicians knew nothing about further and adult education. "The experience of chairing my committee is that everyone made it very clear adult learners are an afterthought."
But Ed Weeple, under-secretary at the Scottish Office Education and Industry Department responsible for further and higher education, said there was "demonstrable evidence" that was not true in Scotland. Fifty-four per cent of FE students last year were over the age of 25 and, while the number of full-time students rose by 2 per cent, the number of part-timers, mostly adults, rose by 23 per cent. In higher education, 22 per cent of students were over the age of 25, against 10 per cent a decade ago. The number of Open University students was up to 12,200 from 7,500.
A further indication of the trend was the number of library users. In the past year, 54 per cent of Scots had used public libraries, the second highest rate in Europe.
Mr Weeple accepted survey findings that Scots were not particularly interested in continuing their learning but pledged Government commitment to changing their perceptions, citing the launch of the University of Industry. "What we are talking about is a second crusade to parallel the new deal" (the four-option programme for those under 25 and out of work).
Baroness Kennedy said: "There are many, many people in our communities who do not have a good experience of school and breathe in failure. We have to provide opportunities for them to reduce that failure. If we do not have a way of making up for thwarted potential, we may pay in the long run."
She said it was vital to invest in the "social capital" of the country and described the near obliteration of community education services as "scandalous".
Charlie McConnell, chief executive of the Scottish Community Education Council, said she was speaking in a city where community education had been "shafted". The lack of a statutory base was the greatest weakness.