Much agree.
One can also set a GPO that disallows specific classes of USB devices. So you can retain the use of a keyboard or mouse, but eliminate a USB storage device. Driver signing is also possible to enforce, along with preventing someone from loading the more popular brands of USB key drivers. If you have your own PKI, you can sign drivers and use your CRL to rapidly whack stuff if needed.
Personally, I'm a big fan of physical security. See also: JB Weld.