You need to either delete them in zwift Custom Workouts or manually add a deleted line to the workouts.files file.
<custom_file>
<name>4Wk_FTP_Booster/Week2/Tempo_with_Finale.zwo</name>
<time>1702588908</time>
<guid>22274</guid>
<checksum>143</checksum>
<deleted>false</deleted>
</custom_file>
By default deleted is set to false. If you delete the workout like you did and set the deleted line to true, zwift will properly delete it the next time it is launched.