Delegates to the annual convention of the 2.4 million-member National Education Association voted two-to-one against the proposed alliance with the 950,000-member American Federation of Teachers, which would have created the nation's largest single union. They spoke of deep-rooted acrimony over membership poaching.
Opponents also complained that the merger would have brought it under the nation's principal umbrella labour organisation, the American Federation of Labor and Confederation of Industrial Organisations, which they argued was predominantly blue-collar.
The defeat for the NEA's leadership comes as teachers' political clout is waning in the face of public discontent with US schools and demands for educational reform.
External critics, mostly conservative politicians, said the vote proved leaders of both unions were out of touch with members.
NEA president Robert Chase said that the delegates had voted against specific aspects of the merger plan, rather than the broader idea. "The dream of uniting America's educators in a single organisation is very much alive," he said.
In what will now be largely a symbolic measure, the AFT is due to proceed with its own merger vote later in the summer.