I checked mine and like @flyrodr I cannot flip the safety up if the trigger is dead.
Taking the slide off it seems like your issue might be in the striker engagement or the safety plunger.
I would take the slide completely apart and add back the parts one by one and see if you can get consistency because something doesn’t seem right currently.
If that doesn’t work then you might have to take your FCU apart. Did you do a trigger replacement or anything with the FCU so far?
Yeah, I’d take the striker and plunger out and give them a close look. Reinstall and then try again.
I tested my manual safety in a boresight grip. If there isn’t play, it’s probably not the issue?