The survey of 5,000 adults raises doubts about the Government's aim to boost the level of qualifications, which is used as a proxy for Britain's economic success and to set qualification-driven targets.
One in four employees said training courses were of little or no value in improving work performance. One in three said studying for qualifications had proved no help at work.
Employees said what they found most useful was help doing the job, being shown techniques by colleagues, engaging in self-reflection and active observation.
Nine out of 10 said they had picked up most of their skills via on-the-job experience. More than half the adults in the survey, carried out in association with the University of Leicester, said "learning by doing" was the most effective way to improve performance at work.
Alan Tuckett, director of Niace, said: "Qualifications are important but they are not the whole story - too much focus on them could be counter-productive.
"We need a well-qualified workforce but getting people motivated and involved requires more and different triggers."
The results show that everyday activities provide the most highly prized sources of learning.
Professor Alan Felstead, co-author of the report, said: "Going on training courses and getting qualifications lays the foundations for initial competence at work, but improved performance depends more on doing the job and learning from others."
The Government had invested heavily in training and increasing the qualifications stock of the UK workforce in a bid to close the productivity gap with other countries.
Professor Felstead said that while this is necessary, the results of the Niace survey suggest the most effective route to enhanced performance lies in improving relations within the workplace, The survey's findings support the Government's decision to give a leading role to the new sector skills councils in designing new training strategies. It shows strong support too for the role of the pound;60 million government-funded union learning scheme.
But the present adult qualification system "goes against the grain" of how people learn at work, said Mr Tuckett in the survey report, Soft Structures, Hard Outcomes.
"We need less cumbersome systems for assessing and accrediting learning and experience gained outside formal study," he said.
However much talk of qualifications motivates planners and focuses those who control the funds, "it does not move learners at work", he said. "As the Employer Training Pilots suggest, too, qualifications are the icing on the cake, when skill, confidence and self-esteem are secured."
Respondents put a host of factors before courses and qualifications. Some 93 per cent said they performed better if they had help to find their own ways of working. The internet proved a poor tool to improve performance - more than half the adults said e-learning was of no help.
Two-thirds (67 per cent) of employees rated paid study leave as "not very important".
It was not surprising, Mr Tuckett said, that people improved most where the employer constantly encouraged staff and managers to keep learning new things.