Gareth Foulkes, education officer for the Disability Rights Commission, acknowledged significant improvements in the past decade. But there was still much to be done in terms of "changing the structures" of colleges, Mr Foulkes said.
Wales had "the highest proportion of disabled people in the UK", but, he said: "I have yet to see a chemistry laboratory here which is accessible to people with a disability."
He said it was futile for politicians and educators to speak of social inclusion if colleges lacked the funds to provide full disabled access.
Although there was increased pastoral and academic support, the real test remained physical access.
"The vast majority of disabled people don't have a physical impairment," said Mr Foulkes. "But as long as a college remains physically inaccessible, what chance does someone with a mental disability have?"
Colleges also needed to develop greater awareness of the 1995 Disability Discrimination Act, amended in 1999.
Since September last year, "service providers" have been obliged to make all "reasonable adjustments" for disabled people, said Mr Foulkes, but in practice this could lead to complex situations.
One partially-sighted catering student was barred from a kitchen placement because she was viewed as a health risk. But the restaurant could not be held responsible under the Act as it was not her "employer".
To prevent such disasters, colleges should liaise more closely with providers before a placement, said Mr Foulkes. Institutions now had an "anticipatory" duty to ensure that disabled students' needs were met. Sites should also be accessible.