A Are you sure your non-contributors aren't just a bit intimidated by the busy brigade? I'm sure A and B teams are the last thing you mean to create,but sometimes where the energy and knowledge level of two groups is very different they have the effect of making each other worse, like a marriage where one is mean, the other a spendthrift.
You will know what I am going to say - that at the first meeting of the year you must review your work-sharing, committees, school attachment rotas, research and preparation tasks. Suggest that you change committee chairs as a matter of policy. From what you say it might be a good idea if you had committee secretaries as well, or a vice-chairman who will also support with minutes and other paper work. Few of us have clerks for more than the main meetings and often committee chairmen in effect run the whole committee - which is a lot of work. Besides, having to write the minutes concentrates the mind wonderfully as you just can't cope if you haven't read all the papers. If you try to pair up a busy governor and a less busy one in these roles it could be a good start. Also, I'm sure you have heard me say that the best decision we ever made was to make our committees open and welcome non-core governors' attendance. As well as building trust and giving you an answer to people who try to re-run a discussion at full governors' meetings, it also increases the knowledge of all governors.
Indeed I found a clue in your letter when you talked about the busy group knowing the school well. Look with special care at whatever arrangements you have for getting into the school on a regular basis, and (yes, using public shame) make sure every governor does it. Often it is feeling inadequate because you are not confident in the school setting which leads to backwardness in taking on jobs. I'll confess something I've never told anyone: I feel inadequate because I don't know the school building really well, and that's not for want of commitment. The bit of my brain that deals with space barely functions and I can't really cope with any buildings much bigger than my own house.
Make sure everyone gets a turn at interviewing for teaching posts: I know no task which makes you feel more responsible. Give individuals assignments which involve talking to someone in the school or working with them. Ask them to present one issue from the agenda now and then.
I don't need to tell you the choice of chairman is crucial. You need someone who is a good team-builder, who can motivate and lead without offending.