I've got to agree.
If I'm just practicing at the range, I can tell the difference between something like CCI 3509 (115gr Blazer) or Federal AE9DP (115gr American Eagle) and my carry load (Federal P9HST3, 124gr +p) as if one was a .22 and the other was a .50 DE.
In drills I can see (very) small differences in splits.
Whenever I'm doing anything that takes my attention away from recoil, I don't notice the difference whatsoever. I use carry ammo for matches and always have going back to the first IDPA Nationals I shot ('00 or '01). The ammo has never been responsible for my performance or lack thereof.
The only meaningful issue is that as your practice & carry loads grow farther apart in terms of external ballistics they'll have different POIs especially at distance. I always zero my gun to my carry ammo and tend to use it whenever I'm shooting at 50yd or beyond. Note that even two loads that appear identical in terms of bullet weight and muzzle velocity may shoot differently enough in a gun that you need to be cognizant of it. For example, I've had 230gr hardball that had a different POI (especially at distance) than the Winchester Ranger and Fed Gold Dot I've carried in .45's.