Having this issue myself since switching from my computer to using the AppleTV. Somewhat makes these free ride workouts pretty pointless.
Please see this thread where I’ve done some AppleTV and Wattbike tests. It still doesn’t work when switching to free rides in a workout, erg mode is not turned off no matter what buttons you then try and press on the AppleTV or the wattbike. Come on Zwift fix the AppleTV app for us