Trevor Phillips said it was time to stop talking about race and start acting.
He told a London conference that inspectors should have powers to crack down on schools and local education authorities not fulfilling their obligations to black, Asian and ethnic minorities.
The former broadcaster said: "We have been talking about schools' ethnic policies all my adult life. I am so anxious we do not make the same errors as we have during the past 20 years, and that because we say we are going to do something we believe it is going to happen.
"We must find a way of compelling schools and if they do not do it they must suffer a penalty."
Mr Phillips said that commission officials would meet their counterparts in the Office for Standards in Education this summer and thrash out plans to ensure inspectors worked towards the maxim that good schools are inclusive schools.
Earlier, education minister Stephen Twigg told the 200 headteacher and council official delegates at the conference, organised by The Education Network, that implementation of the Race Relations Act by local education authorities had been patchy. The Act came into force last year.
The minister said Ofsted would begin assessing schools' race policies in five trial education authorities from September.
* A CRE survey of 3,500 public-sector organisations to assess their commitment to promoting racial equality will be published later this month.