When a previous result is 6 minutes for a segment, and a new result is 3 minutes, that is reported as “50% faster”.
This is incorrect.
The new speed is twice the old speed, and that is 100% faster, not 50% faster. “Fastness” is measured in speed (distance / time). So halving the time (doubling 1/time) is 100% faster.
Alternately, if the former time is 3 minutes and the new time is 6 minutes, that is 100% slower. Slowness is measured in time / distance. So doubling the time is 100% slower.
But the time was 3 minutes faster, which is 50% of 6 minutes. The measure isn’t speed, just elapsed time.
I have to agree with the OP. “Faster” is a comparative adjective regarding speed, not time.
If last night I slept for only seven hours, but the night before, I slept for eight hours, did I sleep “faster” last night? I don’t think so.
Examples:
OP’s math is 50% correct. Halving the time is 100% faster. But doubling the time is 50% slower. 100% slower than any speed is 0 mph.
The percentage comparison is (old time)/(new time)-1.
Examples:
I ate my pizza 3 minutes faster than my friend ate his.
After practicing for the last week my typing is now 10% faster.
The clock in the bedroom is 5 minutes faster than the clock in the kitchen.
My phone loses charge a lot faster in the cold weather.
The older I get, the faster I lose my patience with people.
A few more examples from here https://sentence.yourdictionary.com/Faster
a negative charge is lost much faster than a positive charge.
The more she did for herself, the faster she would heal.
The sand in the hourglass had begun to fall faster the past two days.
Because of Moore’s Law, computers will get faster and storage will be cheaper.
Every time I get close to you, my heart beats faster.
Faster and slower in this context refer to time.
These are basically all comparisons of rates, which are things per unit of time, just like speed is distance per unit of time.
This is a comparison of speed, not time. A comparison of time would be:
I ate pizza on Saturday. My friend ate pizza on Tuesday. Did I eat pizza three days faster than my friend? Earlier, perhaps, or sooner, but not faster.
This is likely a comparison of words per minute, which is a comparison of speed, not time. A comparison of time would be:
Yesterday, I typed for an hour. Today, I typed for half an hour. Did I type faster today than I did yesterday? Well, that would depend on how much I typed, wouldn’t it? Time alone isn’t enough to make that determination.
Back to cycling…
If I am riding at 15 km/h and you are riding at 30 km/h, I think we can all agree that you are riding 100% faster than I am. But, when we get to a segment on Zwift, if it takes me 6 minutes to complete the segment and it takes you 3 minutes, you are now only going 50% faster than I am?!?
I suppose what Zwift is really saying is that you did the segment in 50% of the time it took the last time you did the segment. That is how I have always understood it. Nothing related to actual speed, just 3 minutes divided by 6 minutes equals 50%. Perhaps some different verbiage? “You did that in 50% of your previous time”.
That’s actually not quite right. If it was 99% of the previous speed, that would be 1% faster. If it was @ 0% of the previous speed, ie you did it at infinite speed, that would be 100% faster. This is not how “faster” is typically used, either in physics or in real life. Infinitely fast is infinitely faster.
No it’s not. It’s a comparison of absolute time. There is no mention of speed. I’m 3 minutes faster at eating. What was the speed? You don’t know because there is insufficient information to determine the speed. All you know is I was 3 minutes faster.
This is what Zwift is saying:
3 minutes is 50% faster than 6 minutes. 6 minutes is 100% slower than 3 minutes.
This is what you are saying:
40kmh is 100% faster than 20kmh. 20kmh is 50% slower than 40kmh.
This is what OP wants Zwift to say:
40kmh is 100% faster than 20kmh. 6 minutes is 100% slower than 3 minutes.
Both are correct but as segment times are displayed in minutes and seconds, not kmh, it is only logical that the faster/slower comparisons are used for the segment times and not speed. If Zwift was listing segment position by average speed then by all means you can say 40kmh is 100% faster than 20kmh but they don’t, they use time, so you compare the times.
Measuring the time taken to do a specific thing gives you the reciprocal of the speed at which you can do the thing. (Check the units. Measurements of speed always have time in the denominator, not the numerator.)
If, on my first attempt, it takes me 6 minutes to do that thing, and on the second attempt, it takes me 3 minutes to do that thing, then the second attempt is twice as fast as the first. To substantiate this claim, note that at the speed of the second attempt, I could have done that thing twice in the time it took me for the first attempt. Hence, I was twice as fast. Twice as fast is the same as 100% faster.
In four years of Zwifting, I’ve never thought about nor cared about this.
Huzzah! 
But all of a sudden it seems important.
Boooooo! 