The University and College Union says the loss of up to 350 staff at the Health and Safety Executive and a policy change that will see fewer inspections of workplaces could mean that health risks are not picked up.
Roger Kline, the union's head of equality, said: "They say there is too great a burden on businesses, so the number of inspections has been significantly falling.
"Our concern is that in further education, health and safety issues aren't about thousands of big industrial lathes causing accidents, but the issues are stress and excessive workload.
"If there isn't the threat of a bunch of inspectors coming in, many places won't make health and safety a priority."
Research shows that education is among the occupations where people are most at risk of developing mental health problems. The latest figures from the HSE show that every year, 54 in every 100,000 FE lecturers will become ill through stress at work - more than three times the national average.
In education as a whole, 700 out of every 100,000 employees are injured at work each year, and in the last three years, two people have died.
Mr Kline said he accepted that the financial situation in many colleges made it difficult for them to reduce workload, one of the major causes of mental problems. But he said colleges were not showing enough commitment to reducing undue stress, and said few of them had made local agreements to help cut stress-related illness as they were supposed to after national guidelines were agreed last year.
But the Association of Colleges defended its members, saying that new guidance from the HSE had superseded their agreement.
Stephen Green, the AoC's heath and safety consultant, said: "It's a stressful thing, putting yourself on the line in front of a class - it's the intrinsic nature of the job.
"Workload is one factor among others. But when members of staff feel they are in control, they get an improvement across the board. Colleges are addressing these things as well as workload."
HSE spokesman Ray Allger confirmed that the safety watchdog was consulting over a possible 350 redundancies.
He said a low-regulation approach could be more effective, because the number of people dying at work each year had fallen from 600 to 212 over the HSE's 30 years of existence, while the amount of regulation had been cut by 85 per cent.
But he added: "We have got a range of enforcement powers. We have served improvement notices in relation to stress and prosecution is an option open to us."