Joe Elliott flinches at the memory of his grandfather’s 1947 remark: “We ain’t got no money to buy a bus for your nigger children.”

Thus spoke Roderick Miles Elliott to black parents in Summerton, South Carolina. At that time, he was chairman of Clarendon County district 22 school board and the black parents wanted transport for their children, who were forced to trek miles to reach Scott’s Branch high school while their white peers used one of the board’s 30 buses.

The blunt rejection emboldened the petitioners to sue the schools chief for funds to bring their dilapidated, ramshackle schools into line with those enjoyed by their white neighbours.

When the lawsuit was dismissed on a technicality, the parents mounted the first legal assault on segregation in US schools. The case of Briggs v Elliott took the issue all the way to the Supreme Court, which ordered the desegregation of classrooms in the landmark 1954 Brown v Board ruling.

Fifty years on, Clarendon is still marked by deep racial divisions and the case still rankles.

Shops owned by whites occupy the prime real estate on Summerton’s Main Street, while in the adjoining Railway Street lies a cluster of black businesses. The races are still educated separately, too - whites at the private Clarendon Hall, one of thousands of “segregation academies” founded in the Deep South in the 1960s to avoid integration, while blacks attend the local state school.

In the midst of such division, the man whose surname will forever be associated with past bigotry is on a mission. He speaks out with descendants of those first petitioners in a bid to promote a new reconciliation between the races.

But Mr Elliott, 64, a retired teacher, concedes that reconciliation is unlikely until the white community acknowledges the bravery and suffering of those who took a stand against institutional racism and faces up to its place in the South’s dark history.

“When they reject the courage of the petitioners and the significance of the case, it translates into a rejection of the African-American community,” he says.

“It’s seen as a victory for blacks. It wasn’t. It was a victory for democracy.”

He is not seeking to rehabilitate his grandfather. Roderick was a “product of his times” and the grandson admits that “he’s not the hero in this story”.

But many locals are not ready for his message. Indeed, rather than clearing up champagne corks after last week’s commemoration of the 50th anniversary of school desegregation, Mr Elliott had to sweep up broken glass after his home was vandalised - probably, he says, by “white kids”. Old attitudes die hard.

He is an unlikely avatar of tolerance. The descendant of a well-to-do South Carolina family which can trace its lineage to the third white person born in the state, in 1675, his family tree includes state governors and high-ranking figures in the American Civil War, including an acolyte of General Robert E Lee, commander of the Confederate Army which fought for the South’s right to maintain slavery.

From 1998 to 2001, Mr Elliott was principal of Clarendon Hall school, but much of his teaching career straddled the state’s racially polarised education system. He was also head of an integrated state primary school and at Clarendon Hall he organised a basketball match with Scott’s Branch high. It was the first time many white students had stepped foot inside the local state school.

His connection with the Briggs v Elliott case goes beyond his kinship with the defendant. Harry Briggs, the black filling-station attendant who was pitted against his paternal grandfather as the leading plaintiff in the case, was born and raised in a cabin on the grounds of the lavish cotton-plantation house which Mr Elliott now calls home. And Harry Briggs’ mother worked in the house as a beloved maid for 30 years.

The sacrifice of Mr Briggs and the other petitioners was monumental, says Mr Elliott.

“When petitioners’ names were passed around stores, blacks could no longer purchase from them,” he says.

“Most of them travelled by mule and wagon, so they couldn’t just get in a car and go elsewhere.”

When he was fired from his job at the filling station, Harry Briggs moved to Florida to look for work. Mr Briggs’ wife, Eliza, was sacked, too, from her position at a local hotel.

By acknowledging such “heroes”, Mr Elliott hopes that Summerton might one day exorcise its ghosts and find redemption from its troubled past and its still-uneasy present.