Children could have achieved the target score in the tests with little scientific understanding, as the answers were often included in the questions, an expert has claimed.
Alan Smithers, professor of education at Buckingham university, was sent a sample of 11 questions from all the papers by The TES. He said most could be answered using common sense and comprehension skills.
He said: "They are very easy because you can answer most of them by common sense. They embody too little science. That's unfortunate, because it suggested the tests are not setting sufficiently ambitious goals. It also misleads us nationally as to the level of understanding pupils have."
The Association for Science Education concern said it was concerned that a move to emphasise scientific skills, rather than knowledge, in questions, may have gone too far. However, Colin Osborne, of the Royal Society of Chemistry, said it was right that the tests emphasised investigation skills.
The row comes eight months after ministers hailed record results in last year's tests as evidence that their teaching strategies were working.
Key stage 3 pupils took two one-hour papers on Monday. Each took two tests aimed at levels 3-6 or two at levels 5-7. The number of questions on each paper ranged from 14 to 17. In last year's tests, pupils needed only 57 per cent on the easier papers, or 32 per cent on the harder, to reach level five, the Government's benchmark.
A source at a leading scientific society, who did not wish to be named, said he shared Professor Smithers concerns. He said: "These tests should be about focusing on what students should be getting out of science. The key stage 3 tests do not do that."
In one question, on a level 3-6 paper, pupils were shown a picture of two children investigating the effect of temperature on how fast oil flows through a funnel. One of the children was pictured with a stopwatch. For the first mark, pupils were asked what equipment the children had used to measure the time.
The remaining four marks could be achieved from a basic understanding of how to set up an experiment and read a table of numbers.
Another question, for higher and lower-tier pupils, presented a graph plotting levels of intestinal cancer in six countries against the average amount of starch eaten per day.
Pupils were asked which country had the greatest proportion of people with the cancer, which could be read from the graph. They were also asked what to conclude about the "effect of eating starch on getting cancer of the large intestine". (The graph shows that countries with higher starch intake have lower levels of cancer.) Almost all of the questions required a short answer. If the information could not be deduced from the question, in most cases it was presented in the form of a multiple-choice selection.
The tests have changed in recent years, to put more emphasis on inquiry skills such as how to set up an experiment, and less on factual knowledge.
Derek Bell, chief executive of the Association for Science Education, said:
"My concern is that, by increasing the proportion of the papers devoted to skills, you do not go to the other extreme and set a test devoid of knowledge".
A National Assessment Agency spokeswoman said: "The KS3 test is a test of scientific understanding. Questions have been extensively pre-tested... and are designed to cover the content of the programme of study."