He agreed to hold a ministerial review into the proposed closure of Wirral Metropolitan College's site in Withens Lane, Wallasey, before the election.
But this week he wrote to campaigners saying he could not intervene in the affairs of a college unless it had acted unreasonably or unlawfully - something Mr Blunkett said could not be demonstrated.
Members of the Wallasey Further Education Action Group claimed the closure was needed because of mismanagement at the college. But Mr Blunkett wrote: "Although such rationalisation may well be uncomfortable, it is for the college to manage and develop its estate so as to promote efficiency while preserving responsiveness to local needs."
John Pennel, chairman of the Wallasey branch of lecturers' union Natfhe, said the letter "could have come from Gillian Shephard" and said staff were pressing MP Angela Eagle to sponsor a Private Member's Bill to return colleges to local council control.
Mr Pennel said: "What this means is that when communities want to stop college closures they will get no support. What we find incredible is that there is a whole area of education over which the Secretary of State has no control. "
He said campaigners were angry that Wirral principal Jenny Shackleton had been given a seat on Labour's Welfare to Work task force.