The Government's exam watchdog has carried out an evaluation of the papers taken by 1.8 million seven, 11 and 14-year-olds last May.
Responses from 540 schools to the Qualifications and Curriculum Authority revealed that staff at all key stages were worried that ethnic-minority pupils struggled with the language.
Teachers working with seven-year-olds who had English as an additional language feared a poor test result in maths could "reflect a language problem rather than an accurate measure of maths ability".
At key stage 2, teachers said some of the questions in the non-calculator test were inaccessible to EAL children.
Some maths questions were "too wordy" in the tests for 14-year-olds.
Concerns were also raised about the complexity of some questions in the science paper.
Chris Hassall, headteacher of Taylor Road primary in Leicester, where 90 per cent of pupils have English as a second language, said: "The argument is that maths does not exist in a vacuum, so a problem will be set in a real-life situation with two or three stages. But some of my children cannot get past the first stage because they do not understand the vocabulary."
The rules governing national tests allow some EAL children to have more time. In maths or science, a teaching assistant can read out words. It is restricted to children who have help in class and the adult should only assist with an occasional word at the pupil's request.
A QCA spokesman said: "The language used in national tests is carefully scrutinised to ensure that it does not present a barrier to children demonstrating their true ability."