Kicking kuh" (as opposed to "curly cuh") illustrates that a sound may be represented in more than one way.
A useful rule of thumb for very young children is that c tends to appear earlier in words, k later (often at the end). One-syllable words featuring a single vowel end in ck (for instance, back, neck, stick, clock, duck), one-syllable words featuring a vowel plus another letter end in single k (for instance, ask, seek, hawk, look, think, talk).
Most children easily remember the common exceptions where k begins a word. For this we have the Romans to thank: they decided to do without k, and just use c. The k pages of the dictionary are therefore very short on words from Latin and the Romance languages, which were the main influence on English between 1066 and the 18th century.
There are a few memorable homely Old English words such as king, kiss and kick. In some, like knit and knee, the k has been silent since the 16th century (before then it was pronounced, and for spelling purposes round about Year 3 it helps to revive this tradition). Other silent ks come from Old Norse (knife) and Dutch (knack and knickers).
Otherwise, apart from a few words from Greek roots (kaleidoscope, kinaesthetic, kleptomaniac), initial k leads us to a host of exotic borrowings, like kiosk (Turkish), kebab (Arabic), Kremlin (Russian), kosher (Yiddish), kibbutz (Hebrew), karma (Sanskrit), khaki (Urdu) and kedgeree (Hindi). From the other side of the world come ketchup (China), karate (Japan), kapok (Malay), kayak (Eskimo), kiwi (Maori) and the ultimate "kicking k" word: kangaroo (Aboriginal).