I hope not. It is the duty of anyone writing a reference to ensure that it is fair and accurate, so unless this teacher can show that you have seriously and deliberately erred, you need not feel threatened. What you cannot protect yourself from is his anger and sense of injury.
Some LEAs have decreed that all references should be available to their subjects and such references should make that fact clear when they are sent. If they are stored on computer, they are, in any case, available to the individual under the Data Protection Act.
I have always believed that it is good management to discuss a reference with the person concerned before it is sent. That way, errors and omissions are rectified. It also facilitates discussion about the nature of the job being applied for, enabling the head more accurately to direct relevant comments. A head who is not prepared to tell a teacher that an application does not enjoy unqualified support is ducking what ought to be an accepted professional responsibility. In a situation where appraisal is routine and professional development a priority, discussions about references should produce no surprises.