The warning follows research into the increasing number of people arriving in the UK from the new EU member states.
Most are from Poland although others are from Latvia, Lithuania, Estonia, Hungary, Slovakia and Slovenia.
Many work in isolated rural areas, especially in East Anglia, where they have found employment as low-paid agricultural workers.
Between May 2004 and September 2005, 293,000 new immigrants from these countries were recorded under the Home Office workers' registration scheme.
According to the Learning and Skills Network, which carried out the research, most are "highly qualified" relative to the jobs they find themselves doing, but their opportunities are limited by their poor English.
Many look for work after arriving in the UK but, increasingly, they are being actively recruited in their home countries by British companies. Most of those questioned said their main reason for choosing the UK was the opportunity to learn English. In many cases, they arrive intending to earn enough money to pay for higher-education courses after returning home, as well as supporting family members who stay behind.
The LSN report said: "The shortfall in funding that will be created if the European Social Fund comes to an end is likely to increase the existing gap between demand and supply."
Many colleges rely on ESF cash to run classes in English as a second language. Current ESF funding for ESOL courses comes to an end in 2006.
The Department for Education and Skills said new arrangements are expected to be in place but stressed ESF funding only accounts for about 10 per cent of ESOLfunding.