The ALI is due to disappear as it is swallowed up by an enlarged Ofsted that will cover learning from nursery to adult education.
Natfhe, which itself is to merge with the Association of University Teachers, says Ofsted may not have the specialist knowledge for some areas of post-16 education.
Dan Taubman, education official at Natfhe, said: "Our members have found the specialist knowledge of ALI inspectors and the support they provide after an inspection invaluable.
"Ofsted is very large and our worry is that the years of specialist expertise in colleges and prisons that have been built up by ALI inspectors could be swallowed up and lost in such a vast new inspectorate.
"We urge the Government to enable the new inspectorate to retain ALI's expertise and distinct approach within it. We also call for the new inspectorate to give equal prominence to all groups of learners."
The ALI, formed in 2001 at the same time as the Learning and Skills Council was established, was responsible for inspecting post-18 education in colleges and work-based training.
As well as being an inspectorate for government-funded courses, the ALI carried out inspections for private firms under contract.