HCM covered the main points regarding overall cartridge length.
.40 was designed to shoot a 10mm bullet out of a 9mm sized pistol, an evolution of the brief and troubled FBI experiment with the 10mm cartridge. As the FBI moved to downloaded 10mm specifications, it wasn't long before the industry realized that the larger 10mm case space was wasted and they could get the same ballistics out of a shorter overall cartridge, and .40 S&W was born. The shorter cartridge allowed for these downloaded 10mm ballistics to fit into pistols designed for 9x19mm.
Generally, there was little interest in trying to put .40 back into .45 sized pistols; they already had 10mm for that, which was the proper length to work in a pistol originally chambered for .45.
As noted by others, the competition community found a purpose for the .40 in high capacity 2011 designs in order to meet Major power factor. I don't think Staccato (the rebranded STI) even makes .40 pistols anymore, but I'm sure one of the numerous 2011 custom houses does.