The move follows a government amendment to equality legislation, now before Parliament, which intends to stamp out discrimination based on gender, race or faith.
Schools currently have the power to expel pupils if they or their families change religious affiliation.
The Government had originally planned to allow such practices to continue.
But on Monday, Meg Munn, junior minister for women and equality, agreed to an amendment from the Liberal Democrats removing proposals which would exempt faith-based schools from part of the Bill.
She also used the Commons debate to announce that the rules on school transport, which make no provision for the children of atheist families, would be toughened up.
All but five of England's 150 local councils provide free buses for the children of religious parents who live a significant distance from their nearest faith school.
But a survey by the National Secular Society found that none offers the same right to those parents who live close to a faith school, but wish to send their children to a school with no religious affiliation.
Ms Munn said: "I agree that it needs to make it clearer that cases of those seeking school transport to non-denominational schools should be treated in the same way as the cases of those who seek transport to denominational schools."
Although these rights will not be enshrined in the Equality Bill, Ms Munn said new guidance would be drawn up by the Department for Education and Skills, which would clarify the anomaly.
Under the Bill, all schools will also have to meet new rules to promote equality between the sexes.
Ministers originally planned to exempt schools from key parts of the Bill amid concern over the implications for single-sex schools and lessons.
In a reflection of the strength of feeling that is likely to be provoked by changes to the legislation on faith schools, the DfES has called a meeting of religious groups on February 1.
But the Church of England said it had never backed legislation which allowed discrimination in its schools on any grounds, other than admissions.
The Equality Bill will not affect existing admissions rules, which allow faith schools to choose pupils by their religious affiliation.