Bill Maxwell has promised to create links between the two countries when he takes up his new post in February. His successor has not yet been appointed.
During his two years in Wales, Dr Maxwell, who is a Scot, has led the move to the new Estyn framework. From next year, inspections and reports will be shorter and place more focus on good practice at schools, rather than their shortcomings.
"Although I am very excited about picking up the challenge of the chief inspector's post back up in my native Scotland, I will be sorry to leave Wales," he said.
"I have hugely enjoyed my time working in Wales at what is clearly a time of opportunity and progress for Welsh education in general and for Estyn in particular.
"I am confident that Estyn is now well placed to realise fully the potential of these recent developments and I look forward to sustaining mutually beneficial links between the Scottish and Welsh inspectorates, capitalising on the ancient Celtic connection."
National Association of Headteachers Cymru director Anna Brychan said she was "saddened" by Dr Maxwell's departure.
"We've known for a long time Scotland wanted him back, but we are disappointed."
As well as the teacher-led inspections, schools will also have to complete their own "effectiveness framework" to improve performance. Ms Brychan said Dr Maxwell's efforts had meant the two schemes did not pose too much of a burden on heads.
Gareth Jones, secretary of the Association of School and College Leaders Cymru, said Dr Maxwell had been "a breath of fresh air".
"He had the kind of opinions which were rare when dealing with school inspections - partnership-working and self evaluation," he said.