This hasn’t been my experience in group rides so I’m left wondering which group you are riding with. If I saw a general problem with the level of chat (not just one specific idiot) or a problem involving the ride leader, I would probably contact the Zwift events team so they can review what happened in a specific ride. The low paced group rides I do such as BMTR, Phoenix Rising, RO4H, Team Valhalla, are generally full of lovely people who tell very bad jokes but are otherwise trying to support everyone in the ride.
There is also this feature request that might interest you: