He remembers marching a group into a pub, only to be told by the landlord to leave. "A lot of the other people who were drinking in there actually walked out with us, in protest," he said. "It shows that people will support you if they see the value of what you're doing."
As the adult skills minister, also responsible for youth, he has just seen Connexions, the advice service for young people, returned to his portfolio.
This development has been greeted with relief in some quarters of the Department for Education and Skills, where there had been some dismay that it had been taken away from him in the first place.
Another minister, Stephen Twigg, took over Connexions in June before it was returned to Mr Lewis by Charles Clarke after he became Education Secretary.
Mr Lewis's charity work has taught him about the stress of providing services on the ground when there is uncertainty about funding.
"I checked the mail one day and there was a cheque from a charity for pound;8,000 and a commitment to a further pound;8,000," he said. "I can't tell you how emotional it felt to know we were going to be okay - just because of this money."
So, do ministers and their advisers suffer from a surplus of Islington intellectuals and a drought of experienced practitioners?
"I never lived in Islington," he said. "Sometimes there is insufficient recognition of the skills and life experience of many politicians when decisions are made about allocating responsibilities.
"When I got elected, very little was done about finding out what I have to offer. I think it's getting better and my experience of the civil service is that they are committed and passionate and want to make the system work."
Ahead of him is the job of making sure that Connexions can prove it is having a consistent impact across the country.
Connexions Partnerships provide a network of personal advisers to help young people to navigate their way through anything from educational choices to problems with drugs. There is plenty of evidence of young people's lives having been improved, although nobody - inside or outside the DfES - denies that provision is patchy.
Mr Lewis says local Connexions directors must make sure that quality is at a consistent level across the country if they want to avoid the department intervening more directly.
"I have confidence in Connexions because it is about treating young people with dignity and respect," he said. "I know from my own experience outside politics that this works.
"My job is to bridge the gap between good ideas and reality for people delivering and at the receiving end of the education system.
"As an MP, in your constituency you find out about the impact of policy in people's lives. Charles Clarke has what he calls the Norwich test. He uses his constituency experience to influence the development of policy.
"The test is, does the average person in everyday life feel things are better? We have got to reduce the number of young people not in education or employment. Connexions also has a vital role in reducing juvenile crime.
"Connexions has three years to deliver. Otherwise, I will not hesitate to intervene and change the structure. The Connexions Partnerships must not still be seen as a glorified careers service.
"If, at the end of three years, they have not succeeded in that approach then they will have failed as far as I am concerned.
"We cannot have a situation where so many young people don't get what they need from the education and training system."