Present company excepted for purposes of this discussion
IMHO and experience, the vast majority of people who use the word "niggardly" in the context of stingy or miserly, know exactly what they are doing. It is the second cousin of somebody saying well if "they" use the N-word, and not mean it badly, then why can't I?
I would note that those who have used that word, even presuming no ill intent or controversy, have not done so in a majority black setting i.e. church, school board meeting etc. If they say they do, let's see the video or it did not happen.
FWIW/YMMV Greatly.