Gotcha. That is odd...
I agree, and I wonder if you're trying too hard to control it, and it's causing you to impart some odd directional forces on the gun. Maybe try loading the gun with a single round (in case you drop it) and holding it absolutely as lightly as possible to fire a round. Some slow-mo video (pretty easy to get with an iphone these days) and maybe even the Coach's Eye app would tell you a lot i bet. How's the recoil cycle when you shoot 2-handed?