The P320 is completely modular within the 9/40/357 series pistols. You can mix and match any slide length and any grip module length. There may be a gap between the nose of the slide and the dust cover if you run a longer slide on a shorter frame but it will still work.
There are only three lengths of grip modules - full (17 rounds in 9mm), compact (15 rounds in 9mm), and sub compact (12 rounds in 9mm). There are four slide lengths - full (4.7" barrel), compact (3.9" barrel) subcompact (3.6" barrel), and X-Five (5" barrel). The "Carry" is just the full length grip module with a compact slide. All of these parts will mix and match just fine.