In the middle ages when garbage and waste was thrown out the window the men would walk on the outside of the pathway with very large hats and the hats would catch whatever would be thrown out of the window so it would not land on the lady they were walking with and so when they went inside they would take their hats off and dump everything out.
Removing one’s hat has been a sign of respect since medieval times. It began by removing the hat in church to show respect for God.