But such attire is not out of place at rugby internationals in Corstorphine, Gorrie's home and political stamping ground. And the Liberal Democrats' education and local government spokesman is not hidebound by his privileged background, private school education or family affluence. "A lot of the best radicals come from middle class backgrounds", he says.
He finally wrested the Edinburgh West seat from Tory hands after five attempts. The MP is recognised as a decent sort, a campaigner never out of the local limelight and a party loyalist who gives unstinting service.
Like his parliamentary predecessor, Lord James Douglas-Hamilton, Gorrie excelled in sport at Oxford. Lord James was a boxing blue; Gorrie was president of the athletic club and once held the Scottish half-mile native record.
University was followed by a spell at Gordonstoun, where he taught "history if possible". Then he was head of physical education at Marlborough College. Gorrie says independent schools had a theory you should teach things you knew nothing about.
As a councillor in Edinburgh and Lothian, Gorrie was a constant irritant to the governing party - usually Labour - for 26 years. The experience gave him a rhinoceros shell. Some call him pompous.
Elizabeth Maginnis, the city's education convener and long-time opponent, is not a fan. "He has an obsession with detail that clouds his vision of the bigger picture. And he has a very conservative view of education. He has always carried the banner very well for the middle classes,'' she comments.
Astrid Ritchie, former Conservative education convener, admires his sincerity and dedication to local government. But she says: "The last person to twist his elbow was the last to influence his decision."
Nevertheless, years of shipping abuse in council chambers have prepared Gorrie well for Westminster. He is not into "clever dick debating," and describes the London parliament as "absolutely awful. The speeches are far too long, boring and irrelevant''.
Student funding apart, it is sometimes difficult to shine a light between Labour, Liberal Democrats and Nationalists over education policy. Gorrie feels the essential difference is over funding. He maintains his party and the SNP are sustaining the old Labour traditions New Labour has ditched.
"There's a strand in the Labour party which is anti-teacher. I think a lot of them had unhappy experiences at school, and I find that unattractive. The main difference is that they're not prepared to pay for education. Labour will preside over the worst level of funding for education in real terms for a long time. They are sticking unnecessarily to Tory spending plans and Treasury rules,'' he says.
Gorrie and his party would have no qualms about levying extra taxes on those who can afford to pay.
Two weeks ago Gorrie received a stinging rebuke from his local party that punctured even his rhino skin. A lifelong devotee of a Scottish parliament, he was knocked back for a seat on his own patch.
At 64 he was too old and the wrong sex.
He hopes to creep in via the backdoor party list. But this pillar of the Liberal establishment has himself become a casualty of changing times.