Your holster should have enough tension to prevent someone from easily drawing it. I found out the hard way. We were in a roll over accident and my pistol came out of the holster. I was carrying IWB and had my shirt tucked in, so the pistol did not go anywhere. It did feel a little funny trying to get the pistol back into the holster without pulling my shirt out of my pants, especially with a bunch of police officers around. You just never know when you may end upside down. This was less than a week after my wife had given me a Glock 22 for Christmas. When I informed the officer that I had a concealed carry license, he ask where the firearm was and seemed to be happy that it was not somewhere in the upside down vehicle.
That's pretty crazy. You just never know what may happen.