FWIW, I’ve used a PC, iPad, and now AppleTV for Zwift.
The Gen2 Siri Remote is somewhat better than gen 1 but, yes, anyone who’s used it to navigate the Zwift UI (if they’re being honest) has to admit it’s trash compared to how it works with the rest of the AppleTV. There’s speculation this is because Zwift was designed for a touch-interface and was ported over to ATV.
The mitigating factor for me: I ONLY need it to get riding. After that, I’m using the Companion App on a smartphone:
https://forums.zwift.com/t/phone-tray-add-on-for-tacx-tablet-bracket/
I started with a Tacx tablet tray (because I was using an iPad), but have since gone with the TrainerTray.
The reason I stick to it is speed. It launches in about 10 seconds and “just works” - I don’t think Zwift on ATV has crashed on me in, well, ever, maybe.