John Andrews, who has signed a "gagging clause" preventing him from disclosing details of his departure, is thought to have gone after a scheduled council meeting last Saturday.
In a joint statement with the union, Mr Andrews admitted to a "growing tension" between himself and colleagues over the future direction of the 40,000-strong PAT.
He said: "This is a tremendous wrench after over 25 years working for PAT in one capacity or another, both honorary and salaried.
"Even though I was re-elected last year totally unopposed - the first time any PAT general secretary had been returned unopposed - there has been growing tension between the PAT council and myself over the acceptance and implementation of changes."
Mr Andrews said that he had begun to "develop the association as a modern trade union". But he added: "The PAT council clearly now has different priorities from mine."
After taking over from Peter Dawson in 1992, Mr Andrews reorganised the union, leading to improvements in services for members. PAT's influence on pay talks and other issues remains relatively weak, although its relationship with the other teaching unions has improved. Its chief selling point in the 1970s and 1980s, that it is a no-strike union, has less relevance today.
The details of Mr Andrews' departure are not clear, but it is known that he angered some delegates at his union's conference in Glasgow in July over voting procedures at the annual general meeting.
It is understood that Mr Andrews advised that proxy votes could not be used in a ballot to determine constitutional changes, which were to create separate sections for Scotland and the further and higher education sectors within PAT.
However, members from the Scottish delegation consulted with the Commissioner for the Rights of Trade Union Members, who examined PAT's rules and found that proxy votes could be used for absent members.
At the time, one Scottish delegate told The TES: "One has to ask the question why we are paying the gentleman a great deal of money to know our constitution if he gets it wrong, and then doesn't have the grace to apologise."
The general secretaries of the other unions expressed surprise at Mr Andrews' sudden departure, claiming that he had taken part in a meeting with the Education Secretary, David Blunkett last week, and given no indication he was leaving his job.
In the statement, PAT national chairman, Jane Reid, said: "I am very sorry that John Andrews will no longer be our general secretary, and acknowledge the quality and depth of contribution he has made to our association over a quarter of a century."
David Jones, PAT's deputy general secretary, will take over Mr Andrews' role until a permanent successor is found.
He said: "I have two ambitions, to hand over a vibrant association to the new general secretary, and to maintain and enhance the high quality service we provide to members and the continuing growth in our national influence. "