Unison, the public-sector union which campaigned for 15 years to get the asbestos removed, claimed that when work was carried out at the Hampshire college "pupils and staff were kicking the asbestos dust around like snow unaware that the powder on the floor was dangerous and could have deadly consequences years later".
The Health and Safety Executive decided to prosecute after college staff complained they had been exposed to the deadly dust from 1993 to 1999.
Fareham College pleaded guilty on Monday to four charges of alleged health and safety breaches concerning asbestos control, and was ordered by Fareham magistrates court to pay pound;23,000 in fines and pound;18,000 in court costs.
An HSE spokesman said: "There is no safe level of asbestos. Colleges should just assume it is there."
Caroline Villiers, regional officer of Unison, said: "I think it is important people realise that colleges are riddled with it.
"Staff at Fareham were required to drill and cut into these walls on many occasions without being instructed to wear protective equipment or to warn the members of the public around them.
"There are now concerns not only for the welfare of the staff but also for past students and site contractors who were undoubtedly exposed to asbestos dust.
"These people have to live with the thought that every cough could be the start of this fatal illness."
Malcolm Charnley, the college principal, said: "Unfortunately the college cannot change history, but important lessons have been learned and rigorous systems and procedures have now been put in place."
"The college co-operated fully with the HSE, and also offered staff access to counselling, medical advice and health screening."
Mr Charnley claimed the college would not be affected by the fine as money had already been put aside.
The Further Education Funding Council gave the college pound;300,000 in 1999 to pay for a full survey and asbestos removal. The disease asbestosis can take 25 to 30 years to develop.