Winding was born May 18, 1922 in Aarhus, Denmark. His father, Ove Winding was a naturalized U.S. citizen, thus Kai, his mother and sisters, though born abroad were already U.S. citizens. In September 1934, his mother, Jenny Winding, moved Kai and his two sisters, Ann and Alice. Kai graduated in 1940 from Stuyvesant High School in New York City and that same year began his career as a professional trombonist with Shorty Allen’s band. Subsequently, he played with Sonny Dunham and Alvino Rey until he entered the United States Coast Guard during World War II.