On my M&P, PPQ and H&K's the mag release levers are a polymer/steel combination.
The paddles/button are polymer but the surface that actually engages with the steel magazine body slot is steel.
On Glocks the entire mag release is polymer.
When the sheetmetal magazine body is repeatedly bounced down onto the top of a polymer magazine release by the action during cycling, the knife edge of the sheetmetal magazine slot cuts into the polymer mag release which starts to cause assorted issues from the mag release being very difficult to press to feed issues because the magazine isn't held at the right height anymore.
Plastic Glock magazines used with a steel or aluminum mag release have the opposite issue, the mag slot gets chewed up and the magazine won't stay in place.
It's not the polymer frame that's the issue it's the construction of the magazine release.