Please let me ask this provocative simple question because of the disastrous update yesterday: Does Zwift have any kind of Quality Assurance for their software products?
If yes, why was it possible to let this crappy update “destroy” Zwift’s customers setups?
If no, why not?
I’ve seen job listings for QA engineers at Zwift, but that was before the layoffs last year so they could all be gone now 
.
To be honest I doubt Zwift saw this in their testing. If I was doing QA on the 3 systems that I have for Zwift I would tell you there is no issues with ERG or Incline. 3 windows setups with very different hardware no resistance issues.
This didn’t come up in any of our “play testing” or the significant amount of QA time in testing the routes.
I think it’s very clear they didn’t. If they had it would never have seen the light of day until such a showstopper was resolved.
Maybe, but in terms of smoke testing (or any QA/testing other than for a specific platform), this wouldn’t be a sensible approach even with different trainers. More likely QA is conducted across a variety of platforms. 
With all due respect, what did you guys test? Is there a test plan, or any automation? Broken drops is a bummer but it’s not game-breaking. “ERG mode doesn’t work” is game-breaking for many people and should’ve been immediately obvious if it had been tested.
It might not have been a problem on any of the configurations they use for testing.
The thing is, not everyone has these issues, as the update thread posts show.
I am just pointing out that on 3 windows Systems resistance worked as intended so it could be that in all the windows tests that Zwift did they found the same as what I did.
Going by the people I know personally, I think about 1 in 12 had any of these issues. Don’t know exactly what system combos they represent, but I do know there are Win 10, Win 11, ATV, and Android users in there, BT and ANT+. That’s of course the smallest of data points (‘people I know’, lol). But yeah, a lot of people didn’t see any issues at all. Without knowing the number of users who experienced issues, and the number who did not, it’s impossible to say what the odds were that Zwift testers didn’t see issues. Could have been unlikely, could have been very likely.
With respect, we’re not going to publish our test plan in an open forum. But please believe me when I say we were testing the Scottish map (and associated release) for months.
It would be super interesting to know what ZHQ found, because this is the weirdest part to me.
I have two teammates running the same setup as I am; ANT+ on a Kickrbike on Windows 10.
I’m the only one that had the issue; out of my whole team for that matter.
I’m still also quite angry about the whole situation, but, being as spotty as the problem is, I guess it makes sense.
If anything is learned here; ZHQ needs to have a public beta program; how they want to run it is up to them.
But as others have stated, this patch literally made the software I’m paying for, completely pointless.
Not that I’m pointing any fingers at anyone for the whole situation; but… it’s a dangerous situation to be in, where the core purpose of your software… fails to function, that’s the real issue here.
For sure–I don’t want to minimize the problem for anyone. Even if it was just a workout platform for individuals, a product that’s not working would be an issue. But Zwift involves racing and teams and that means there are obligations and other people affected. It’s not ‘silly’ to be upset by the software not working. It’s definitely a bigger problem than a KOM that counts backwards from the top (cough Rooftop cough), because the software still works in that case.
A beta program would be a good idea, for sure. I’d suspect they could get people to sign up for nothing more than the perceived ‘cool factor’ of being part of the beta team, too. Would that have caught this? Odds would have improved, given that a beta group+internal testing would be larger group than internal testing alone. Is it needed before we can say whether they did their job? Without knowing the extent of their testing protocols, I don’t know how to tell.
I don’t know what standards are anymore, if there are any, for customer redress. Spotify was out for almost the whole 2.5 hours I was working out the other day, which honestly did affect my workout a bit. And had I been relying on it for something like podcast listening for research for a job, it could have been worse. I doubt Spotify is going to knock anything off my bill for the next month, not for 2.5 hours. At what point, how long of an outage, is needed before people start thinking they should have to pay less because of it? (Probably will vary wildly–2.5 hours for some, maybe 
)
There’s a natural “I shouldn’t have to pay for this” feeling, but I think there’s also a recognition with a piece of software that it might not always work properly. (Which is not a concession we make for things like microwaves or fridges…if my appliances stop working for a day, I want some monetary compensation! 
)
Do we even have an SLA with Zwift? I think not; at least as far as I recall. And to be fair Zwift’s uptime is generally very good.
The Terms of Service (section 14) are quite clear that unreliability or problems are within the service agreement I’d say, so compensation for an outage would be unlikely to be forthcoming other than as goodwill.
Zwift does not represent or warrant that our Platform is accurate, complete, reliable, current or error-free.
I didn’t have the resistance issues that others have, so it’s a matter of different systems and scenarios to replicate the problem. So it’s not quite so easy to point at QA and level blame at them.
Drops issue yes, that’s fairly obvious to spot - sometimes things can slip by even with best efforts to stop it.
They will get it fixed.
What’s better?
A stable, but stagnant training application with no new features ever added, or one that continuously adds features, like new roads, pace partners, holo replays, and some bugs along the way?
That would represent completely inadequate testing. And that’s normal for an individual user - you don’t represent the appropriate test suite. The proof in in the outcome. Many users with common equipment are affected.
I didn’t say I represent the QA department but if they had 10 windows PC’s and none had the issue how would they know.
I also don’t want bugs slipping through just like you, I hope that they can learn from this and improve testing even more.
This is an epic failure no matter how you explain it. The testing regimen is inadequate and has been since forever. People with perfectly commonplace equipment were unable to do workouts. That’s all you need to know. The testing has to deal with perfectly commonplace equipment or it’s a failure.
A system where there is a week’s grace period before updating is mandatory.
And don’t tell me it can’t be done because it is done already.
Apple iOS and tvOS (and I presume Android users) are not forced to update immediately. They are usually given a few days before they must update. Perhaps mainly because Zwift can’t control exactly when updates are released on app stores. Imagine if you couldn’t log in to Zwift because you needed the latest version but you couldn’t download the latest version because it wasn’t in the app store yet.
A similar system for PC and Mac users would allow those who want to be unpaid beta testers for Zwift to update at their own risk while the rest of us can wait until the inevitable bugs are ironed out a week or two later.
It’s unreasonable of users to expect completely bug free releases but on the other hand it’s unreasonable of Zwift to force users to deal with the abnormally high number of bugs they seem to be producing lately.