Zwift Racing Score Update [October 29 2024]

We’ve been really grateful for the influx of feedback we’ve received since the public launch of Zwift Racing Score earlier this month. We’ve been collecting and prioritizing the feedback and have some updates to share.

More transparency on the scoring system
There has been a lot of conversation about how score changes in race results seem to follow a consistent pattern of the top half of racers getting a score increase, while the bottom half of racers get a score decrease. We’ve decided that the best way to address this is to shed some more light on the mechanism we are using for the scoring system.

Zwift Racing Score is leveraging a customized version of OpenSkill, an open source rating system based on a similar statistical approach that Microsoft uses at scale for XBox Live.

The system works by tracking an average skill and uncertainty factor for each racer, both of which adjust with each race result. To put this in statistics terms, each racer has a probability curve for their skill, and we track the average and standard deviation (aka uncertainty). Finishing near the top of the results tends to result in an increase in average skill, finishing low usually results in a decrease. Consistent results tend to make us more certain of a racer’s skill, surprising results tend to make us less certain. ZRS is this skill probability boiled down to a single 1-1000 number.

We aren’t just using OpenSkill out of the box, we have added several customizations to adapt it to Zwift’s specific use case which needs to consider physiological ability in addition to skill. We also transform the unbounded skill from OpenSkill into our bounded 1-1000 range.

So, why did we choose OpenSkill? We evaluated OpenSkill amongst other options like Elo, Elo-MMR, etc. In our simulations and analysis, we appreciated some of the features that OpenSkill provides such as the ability to adjust individual uncertainty, support for team-based competitions (something we eventually want), and more tunable parameters to help racers get to an accurate score as quickly as possible.

Removal of 30s power from seed formula
Using 30-second critical power bests in our seed formula has been an ongoing topic of feedback, so we took a closer look at it.

We received reports of people being overseeded from hitting a 30-second power best outside of race conditions—often higher than what’s achievable while fatigued in a race. Taking some community advice, we analyzed mass-start events to reduce bias from pack or sprint finishes. We found that while 30 seconds is still a solid metric, we can do an even better job at seeding by factoring in alternative data points.

As a result, we’re working on an adjusted formula. More details on the rollout and potential impacts on current scores are coming soon. We don’t plan to do a full score reset with this adjustment.

More mechanisms to protect against sandbagging
One of the main goals of introducing ZRS is to make virtual racing as fair and fun as possible. We realize there will always be ways to game the system, but we’re committed to continuing to add measures to prevent it. Next week we’re targeting to enable a mechanism that prevents scores from dropping if we detect a much lower than usual effort from a racer. This is a similar mechanism to what ZwiftRacing.app has in place. Indicators in the race results for when this mechanism kicks in will follow. We’ll also continue evaluating other measures that can be added to minimize cheating.

Decay will be turned off (for now)
The original intention with decay was to account for decays in fitness after going an extended period of time without riding. After evaluating the functionality and reviewing feedback, we have decided to disable decay for now since it doesn’t seem to be providing the intended value. We may turn this back on later after making some improvements to the mechanism. People will see small increases in their scores once it is turned off.

There are more updates to come! Development is continuing and we’re excited to keep iterating on Zwift Racing Score to make racing even more exciting and fun. Thank you for all of the feedback and conversation.

32 Likes

Sounds a positive update.

As regards sandbagging, the triggering of this needs to appreciate that most Zwift races and TTs are standalone events, rather than a series where General Classification is timed based from a number of events.

In a standalone event, once you have been dropped by a group, there absolutely no benefit of continuing to work hard. If you continue to work hard, all you’re doing is accumilating fatigue that will negatively influence races and/or TTs in the short term.

In a series, the sooner you can cross the finish line even if dropped, the less time you give to your rivals.

3 Likes

This sounds very difficult to factor in. I think I’d prefer quite tough anti tanking measures over being able to soft pedal after getting dropped.

The update sounds very positive, I agree.

I’m assuming that for those people whose floor is heavily influenced by their 30 sec power, this will change but their ZRS wont, and for those new to zwift their seed score will be determined by thr updated algorithm. It is reassuring that 30secs is not one of the strongest determinants of position in a mass start race. I’d imagine zmap alone does a pretty decent job…

Thank you for listening and iterating!

7 Likes

Sound like good changes. The 30 seconds did not make sense so good for it to be gone.

Regarding the Open skill methodology: Haven’t had time to read it all but it can be made similar to ELO apparently. However, it seemed much more simplistic when looking at how scores changed (but that was purely observational from posts on forum where half moved up, half down which implied low certainty based on docs). Any insight into how it used would be useful rather than having to guess the chosen values for several parameters. As Zwift Racing uses ELO and the scores seem much more intuitive and based on relative skill levels than ZRS.

Thank you for this interesting update, and for listening to community feedback.

2 Likes

Thank you for sharing this updates.

I really hope we get this updates sooner rather than later…as we are starting our VirtuSlo 4Endurance League 10th November and we still haven’t decided on Pen Ranges we will use. https://virtuslo.cc/

Nothing more to say than kudo’s for these changes. It took a while, but I am glad our concerns finally arrived at ZHQ :wink:

Most of the things I was unhappy about are solved with this update. Only thing that needs to be done is the Companion App and the Zwiftpower page where we need to see the ZRS more clear. And I would really like to see everyones raise or drop after a race and not just a new total for everyone, cause that says nothing to me. Just make it +9 (484)

And ZRS needs to be added to Zwiftpower asap.
And I would like to always be able to see my seed score since that is still an important number. Just put it on the zwift.com page in my profile or something.

7 Likes

This would be nice.

1 Like

:100: This is a useful racing goal especially for people who can expect to finish off the podium. In zwiftracing.app I like to see how far above my zrCS I can get my vELO score. Am I outperforming expectations even though I’m getting dropped?

2 Likes

I’ll echo the kudos for announcing important changes that have a big impact on ZRS’ success.

I have an observation about not doing a score reset though. While the initial seed score was just that, a starting point, it’s important to remember that the seed score truly is the ongoing and perpetual basis for everyone’s racing score. Every result-based increase or decrease has been based on that initial seed score, so anyone with an inflated seed score also has an inflated current racing score.

Hopefully there are plans in the works to apply some kind of adjustment to those whose 30sec power caused them to be overseeded. I think the wording of your announcement leaves that open as a possibility, so keeping my fingers crossed.

Thanks again for listening to the community!

1 Like

Yeah, I think that is correct. Especially for folks who don’t race a lot. If you were way overseeded, and only raced a few times you likely only dropped 10 or so per race, so might not have yet hit where your new seed would have placed you. Actually if they remove (or radically reduce the importance of) 30s power I’m somewhat surprised they don’t re-calculate seeds.

1 Like

Are you still considering increasing the look-back period for returning racers to something longer than 90 days? It’s not just an “it will work itself out over time” thing either. The “returning rider” problem will rear its ugly head for at least 3-4 months every single year unless some assumptions are made that a returning 700RS rider is still a ~700RS rider until they prove otherwise.

2 Likes

I interpreted that as they will recalculate seeds, and as such score floors. But they won’t reset everyone to their new seed. If someone’s new floor is higher than their current score they will get pushed up to that.

3 Likes

I can understand not doing a full score reset because that would lose all of the event-driven changes over the last month. But surely they’ll calculate new seed scores for everyone. So they could compare the new seed scores to the originally-calculated seed scores and subtract the difference (or a percentage of the difference).

1 Like

I’m not intending to question the score rationale, but hope to provide a sample of how it is experienced.

Today I got crushed by a bunch of “B’s” in a crit race, all with racing scores lower than me. Yet I now get promoted to above 520 category. See the attached screenshot.

This type of outcome and corresponding point shift is baffling to me.

4 Likes

Your seed score hasn’t increased, just Racing Score based on finish position.

Finish a race badly and you will drop back into the lower pen.

Crushed? I guess you must have been dropped early in the race and rolled in several minutes behind.

:joy::joy::joy::joy::joy::joy::joy::joy::joy::joy::joy:

2 Likes

The problem that you are highlighting is what they just announced they would change.

Your 30-second power is 10.1w/kg and 916w, which drives your seed score higher than most of the riders around you, including the 2 “B’s” that are C’s by ZRS due to their lower (half your output sprints).

So maybe with the 30 seconds removed, you drop back to C anyway.

2 Likes

You are describing the race before this one and my next race as I represent 521 in 520-690 category :muscle:

1 Like