For weeks now, every shopping street in Scotland has been reeling to the sound of piped carols and ringing tills. Everywhere you look there are glittering Christmas trees, rotund and rosy-cheeked Santa Claus figures, a confusion of reindeer, angels, mistletoe, French hens and choirboys.

The way we celebrate Christmas is a cultural mish-mash, the result of ancient traditions with a long and complex history of change, repression and adaptation. Scottish New Year celebrations too are tied up in this complicated past, yet it is difficult to conceive, as thousands of people descend from far and wide to celebrate “Edinburgh’s Hogmanay”, that the origins of this urban extravaganza lie in the rituals of small peasant communities.

Some of the oldest traceable Scottish customs are those now associated with New Year. The peculiarly Scottish notion that what we do at New Year will set the tone for the next 12 months - New Year’s Day as a microcosm of the whole year to come - is linked to the ancient celebration of the Dark Days. These 12 days at the turning point of winter were, since earliest times, a period when chaos and reversal were celebrated. This was the period when the Abbot of Misrule held sway, when masters waited on servants, when men dressed as women and women as men, when teachers were “barred-oot” of the school by the pupils.

The logic of this has been a mystery until very recently when Emily Lyle of the School of Scottish Studies at Edinburgh University proposed a solution. In societies with a cyclical notion of the passage of time, she suggests, people did not always visualise the year as a closed circle with the end of one year automatically bringing them back to the beginning of the next. The year may have been perceived as a progress from A to B, which demanded some means of “getting back” to A for the start of the next cycle. The 12 days fulfilled this requirement, acting as a kind of “rewinding”, a symbolic return to the beginning of the cycle of seasons.

This symbolic reversal would explain the tradition of unruly behaviour that would have been deemed unacceptable at any other time of year, and the belief that conditions on the 12 days foretold conditions in the months to come - for example if the seventh day was wet, that meant a wet July to follow. The rituals people employed in the earliest days of our history to ensure a good harvest or the safety of family and community have clear similarities with our first-footing and associated traditions. In those days the fire which burned in every household, providing heat, light and cooking needs, would never have been allowed to go out.

Yet for the “needfire” celebration, all fires in the community were extinguished, and a new fire kindled by the friction of two pieces of dry wood. This was the needfire, and all the fires of the community were lit from that flame; water was heated on it and sprinkled on sick people as a cure, or on cattle and people as a means of protection. A needfire was recorded on Mull in the latter part of the 18th century, and the same rituals were found to be alive and well among some African tribes as recently as the 1950s.

Needfire was a means of defining a community, of including everyone in the group, but there are other New Year traditions which appear to split the group, to set one half against the other, and some which set one household against another in the quest for good fortune in the year to come. The ball games which were once a feature of winter festivities across Scotland now remain only in Kirkwall and Jedburgh. In Kirkwall, men born in the north end of town, the Uppies, take on those from the south, the Doonies. The game is played on New Year’s Day, the “goals” are at the harbour and at the “head of the town”, and the object is to “hail” the cork-filled leather ball into your own half, thus hauling good fortune to your side of the town. If the Doonies win, the year will bring good fishing; while victorious Uppies can look forward to a good harvest.

The idea of a limited amount of good fortune which must be won away from someone else is taken a step further in the tradition of Creaming the Well. Those who were envious of a neighbour’s success, particularly if their cows produced good and plentiful milk, would go at midnight on Hogmanay to draw the first water from the well the neighbour used - the cream of the well. The dairy utensils would then be washed in this water, and the rest given to the cows to drink. This was supposed to ensure a good supply of milk and butter, presumably at a cost to the neighbour.

The basic impetus behind these ancient customs has survived until today in the form of first-footing, but in the interim a mass of foreign and homegrown influences have affected our winter festivals. The Roman festival of Saturnalia, celebrated for several days from December 17, consisted of noisy revelry, present-giving, the lighting of candles and some of the role reversal associated with the Dark Days. In Scotland, however, the Roman influence was greatly outweighed by that of the Vikings. The Norse festival of Yule with its fires and feasting had a lasting effect, and by the 13th century, the midwinter festival in Scotland was universally known as Yule.

With the coming of Christianity, the church chose Christ’s birthday for him, and invested the pagan festival with a Christian significance. During the Middle Ages the partying still went on for 12 days, but with peaks of activity at December 25, January 1 and January 6 (Epiphany). At this time, Christmas Eve was a day of fasting. No meat, eggs or cheese were allowed, so the feasting on Christmas Day must have been all the more welcome. For the next 300 years, the midwinter festivities were a time of feasting and games, one of the rare occasions when workers laid down their tools and masters would keep open house.

The consumption could be on a grand scale. At the end of the 14th century Richard II held a Christmas banquet to which were invited 10,000 guests. The party polished off 200 oxen and 200 tubs of wine. In 1561 the Scottish Kirk claimed, correctly enough, that “the Papists” had invented Christmas. So they proceeded to abolish it. For the next century Yule was a battlefield between the Kirk and the people, who were intent on keeping up their traditional Yuletide pursuits.

The north-east of Scotland was particularly tenacious in clinging to its Christmas festivities. In 1574 the Aberdeen kirk session tried 14 women for “playing, dancing and singing of filthy carols on Yule Day at even”. In Elgin in 1599 the populace was forbidden to indulge in “profane pastime . . . footballing through the town, snowballing, singing of carols, or other profane songs, guising, piping, violing and dancing”. Even the king broke the rules. Yule was regularly celebrated by James VI at the royal court in Edinburgh, and in 1600 there was “shooting of cannons out of the Castell of Edinburgh, and other signes of joy”.

In 1617, the Christmas-loving king reimposed the religious celebration of Christmas and other holy days, but the Kirk fought back and in 1641 the total abolition of Yule was ordered. Christmas celebrations in the Lowlands were more or less subdued, but the revelry gradually reappeared at New Year, and the Kirk eventually turned a blind eye. The result was that even 100 years ago Scots children hung up their stockings on Hogmanay and people went to work on Christmas Day.

Thanks to Dr Emily Lyle. The Hogmanay Companion by Hugh Douglas (Neil Wilson, 1993); The Silver Bough by F Marian McNeill (W McLellan, 1957); The Stations of the Sun by Ronald Hutton (Oxford University Press, 1996)