There have been some examples of bathroom sexual assaults by people claiming to be transgender. https://www.wave3.com/2019/03/25/man-arrested-after-allegedly-inappropriately-grabbing-woman-walmart-restroom/ However, I agree that the right blows this issue out of proportion and often uses it deceptively. (I saw a Breitbart piece listing incidents showing that males are a danger to “women and children” in public restrooms that included several incidents of male children being assaulted or filmed by adult males!) And I think there are related issues the gender-critical feminists haven’t really thought out: for instance, they often argue that transgender ideology gaslights a lot of young butch lesbians into thinking they’re trans men and that women should be told they can dress and look as “masculine” as they want without adopting a male identity. I agree, but the same women also argue that women have a right not to be subjected to male-looking people in public bathrooms as a matter of their sense of safety. I know there have been cases of masculine-looking lesbians being mistaken for men in women’s bathrooms and getting yelled at, shoved, or even arrested.

Personally, I think the optimal solution would be more single-user, unisex restrooms everywhere, but that’s pretty expensive so probably won’t happen anytime soon!