Pupils do not enjoy working in teacher-designated groups that separate them from their friends, the national Primary Review has found.
They are also unhappy when pre-determined groupings result in too much or too little being demanded of them in lessons.
This contradicts the Conservatives' claim this week that pupils work best in sets determined by academic capability rather than in mixed-ability classes.
John Bangs, head of education for the National Union of Teachers, said: "Pupils want to be with their friends. Rigid grouping by ability increases the likelihood of too much or too little being demanded of them, which they do not like.
"It makes teachers' lives much easier if pupils are constructively engaged in their learning."
But Chris Davis, of the National Primary Headteachers' Association, does not believe pupils are arguing against setting.
"Work should always be challenging, but within pupils' reach," he said. "That's an argument against bad teaching.
The latest Primary Review report says pupils become demotivated when they cannot see the purpose of their work, when it is repetitive or unchallenging and when it involves a lot of writing and little physical movement.
Jean Boardman, head of Lowercroft Primary in Bury, Lancashire, agrees. Teachers at her school make an effort to incorporate drama and art into academic subjects on a regular basis.
"Every child has a different learning style, so you need a multisensory approach," she said.
"We try at some point each day to ensure that learning activities are fun. Then children enjoy themselves. And if they're enjoying themselves, they'll learn more.
"It's not rocket science."