Jewish males cover their heads with a kippa and/or a hat, as a reminder of the presence of God. This practice is twice mentioned emphatically in the Talmud (Shabbat 156b; Kallah 1:16), in statements dating back 1700 and 1850 years, respectively. Even then, covering one’s head is spoken of as an established practice, not something new.
The Yiddish word for kippah, “yarmulkah,” is a contraction of the Aramaic “yerei malkah”: to be aware of the King.

Note that Jewish married women traditionally cover their hair (Talmud, Ketubot 72b). This is for the purpose of modesty – only her husband should see her beauty – since the hair is considered beautiful (Talmud, Berakhot 24a).