Between mine and my nieces and nephews that live with us a few months a year I've been through six so far.
I really don't think you have anything to worry about.
I play, ride bikes, horses, have kids all around.
If you ever drop your gun, just throw your phone on the ground. They'll go for that 10 times out of 10!
ETA: I haven't had one that interested in a gun I was wearing for more than a few minutes the first time they see it. After that it's just part of your clothes.