Mr Davies, 40, will leave the pound;40,000-a-year post in late September, to become secretary to the broadcasting council of BBC Wales.
The former chief executive of Plaid Cymru has worked for NAHT Wales since May last year. He was being invested at the National Eisteddfod today as an honorary member of Gorsedd of Bards, the Welsh equivalent of a knighthood.
Mr Davies says that his work for the NAHT has often been frustrated by backbiting between teachers' unions over issues such as workload. He said:
"There's so much sectarianism. It really does a disservice to the profession that teachers are represented by seven different unions."
He urged unions to recruit more employees from outside the profession: "It was such a good thing that I came from outside the world of education, with an untainted view. I was able to assess things rationally, which was an enormous advantage. I hope the union will learn from this."
David Hart, general secretary of the NAHT, said: "Karl certainly made a significant contribution, and I'm sad he's going."
Edwyn Williams, general secretary of UCAC, the Welsh-speaking teachers' union, has also handed in his resignation. The 55-year-old, who held the post for seven years, will leave in September. He is to leave shortly before the end of his term of office, as he says that the union needs additional time to choose a successor. Gruff Hughes, UCAC deputy general secretary, will fill the post until elections in November.
Mr Williams, a former head of geography, says that he plans to continue to work as principal geography examiner for the Welsh Joint Education Committee exam board, and may return to the classroom.
He said that a lot still needed to be done in Welsh-medium education, but added: "I hope the classroom will be a better place, because of the work I've done."