Mr Twigg said he was "very disappointed" by the move by the Assessment and Qualifications Alliance board, taken without consultation last month.
Subject associations are furious about the decision, which means that from 2006, only the Oxford, Cambridge and RSA board will offer the subjects.
Mr Twigg was challenged to respond in a debate in the House of Commons, called by Michael Fallon, a former Conservative education minister.
Mr Fallon, backbench MP for Sevenoaks, said the "mean-minded" decision, made without consulting even the board's own subject advisory committees, would drive classics out of state schools.
Comprehensives, he suggested, favoured AQA's GCSE, for example, because it was less demanding on vocabulary than OCR's qualification. State students do not spend as much time studying the subjects.
Mr Twigg said: "I am very disappointed by the AQA's decision. I absolutely share (Mr Fallon's) concern about the failure to consult fully and in particular about the failure to consult the subject associations. The situation could have been handled much better."
He added that he too was worried about the possible impact of the move on the take-up of the subjects in the state sector. Mr Twigg stressed, however, that he had no power to insist that AQA offered the exams.
Thirty-seven MPs have signed an early day motion tabled by Mr Fallon deploring the decision.
Latin and Greek were two of 11 minority exams, including GCSEs in Russian and archaeology that AQA has announced it is withdrawing. The board has put the decision down to small or declining numbers taking the exams.
An AQA spokeswoman said the board would wait until it had received Mr Twigg's letter before deciding how to respond.