We’ve been really grateful for the influx of feedback we’ve received since the public launch of Zwift Racing Score earlier this month. We’ve been collecting and prioritizing the feedback and have some updates to share.
More transparency on the scoring system
There has been a lot of conversation about how score changes in race results seem to follow a consistent pattern of the top half of racers getting a score increase, while the bottom half of racers get a score decrease. We’ve decided that the best way to address this is to shed some more light on the mechanism we are using for the scoring system.
Zwift Racing Score is leveraging a customized version of OpenSkill, an open source rating system based on a similar statistical approach that Microsoft uses at scale for XBox Live.
The system works by tracking an average skill and uncertainty factor for each racer, both of which adjust with each race result. To put this in statistics terms, each racer has a probability curve for their skill, and we track the average and standard deviation (aka uncertainty). Finishing near the top of the results tends to result in an increase in average skill, finishing low usually results in a decrease. Consistent results tend to make us more certain of a racer’s skill, surprising results tend to make us less certain. ZRS is this skill probability boiled down to a single 1-1000 number.
We aren’t just using OpenSkill out of the box, we have added several customizations to adapt it to Zwift’s specific use case which needs to consider physiological ability in addition to skill. We also transform the unbounded skill from OpenSkill into our bounded 1-1000 range.
So, why did we choose OpenSkill? We evaluated OpenSkill amongst other options like Elo, Elo-MMR, etc. In our simulations and analysis, we appreciated some of the features that OpenSkill provides such as the ability to adjust individual uncertainty, support for team-based competitions (something we eventually want), and more tunable parameters to help racers get to an accurate score as quickly as possible.
Removal of 30s power from seed formula
Using 30-second critical power bests in our seed formula has been an ongoing topic of feedback, so we took a closer look at it.
We received reports of people being overseeded from hitting a 30-second power best outside of race conditions—often higher than what’s achievable while fatigued in a race. Taking some community advice, we analyzed mass-start events to reduce bias from pack or sprint finishes. We found that while 30 seconds is still a solid metric, we can do an even better job at seeding by factoring in alternative data points.
As a result, we’re working on an adjusted formula. More details on the rollout and potential impacts on current scores are coming soon. We don’t plan to do a full score reset with this adjustment.
More mechanisms to protect against sandbagging
One of the main goals of introducing ZRS is to make virtual racing as fair and fun as possible. We realize there will always be ways to game the system, but we’re committed to continuing to add measures to prevent it. Next week we’re targeting to enable a mechanism that prevents scores from dropping if we detect a much lower than usual effort from a racer. This is a similar mechanism to what ZwiftRacing.app has in place. Indicators in the race results for when this mechanism kicks in will follow. We’ll also continue evaluating other measures that can be added to minimize cheating.
Decay will be turned off (for now)
The original intention with decay was to account for decays in fitness after going an extended period of time without riding. After evaluating the functionality and reviewing feedback, we have decided to disable decay for now since it doesn’t seem to be providing the intended value. We may turn this back on later after making some improvements to the mechanism. People will see small increases in their scores once it is turned off.
There are more updates to come! Development is continuing and we’re excited to keep iterating on Zwift Racing Score to make racing even more exciting and fun. Thank you for all of the feedback and conversation.
This is a useful racing goal especially for people who can expect to finish off the podium. In zwiftracing.app I like to see how far above my zrCS I can get my vELO score. Am I outperforming expectations even though I’m getting dropped?
