An unarmed man is most assuredly still a threat to life and limb. I've seen people knocked unconscious with a single punch, and even people killed outright from a single strike from a bare fist. Thus, in nearly any confrontation that becomes physical, I can truthfully verbalize fear for my life. The potential is easily there.