Fixing broken activity files
TL;DR:
If you have a broken Zwift activity file which you want to fix and upload to Strava or Garmin Connect check out this tool.
What happened?
On a Sunday morning few weeks ago I made the KISS 100 Race Series (EU)(NO HUD) event.
After 4 laps of Roule Ma Poule and little over 3 hours I finished the 96km event. I did my cool down to round it off to 100km and ended the ride. I clicked the OK button, which made it’s bounce animation
and Zwift froze.
OK, no worries it was a big ride lets give it some time. I took a shower, baked some sour bread,
and after an hour or so Zwift was still stuck. Meanwhile race result was uploaded to ZwiftPower
and companion app, but with broken file that wouldn’t display any information, and yes no upload to
Strava or Garmin Connect.
Here is how it looks in companion app and zwift.com:
At that point I decided to force quit Zwift and look into the local folders for some activity file and manually upload it there. Found two files one 0.6km long and another one that would not upload to Strava or Garmin Connect, or even validate as a .FIT file, but given it’s hefty size of 434kB this should be the one.
I came to the Forum read a dozen of threads describing similar situations after Zwift crash or loss of
Internet connection during upload. the judgement was always the same: ‘If the file is broken there is nothing to be done.’
I tried to do something with garmin online fit repair tool but it gave an error straight away, and Fit File Tools which made the file uploadable to Strava but messed up the data.
At that point all I had was one broken ride in Companion App, and no way to add the ride to my training score over at garmin connect, or brag about it on Strava. I had done 3 hours and 100km on a trainer and had nothing to show for it. Not cool, not cool at all. So lets turn on uber-geek mode and see what … can be done.
The solution
After some investigation this is what I found:
The file is broken, because it is unfinished. It has invalid header and after some data records it ends abruptly. To be considered a valid FIT file it needs to have correct header, some data records and a CRC at the end. Moreover to be uploadable to Garmin Connect it needs to have a summary section, which was again missing in this file.
The good news is that all of those things can be calculated from the existing unfinished file and appended to it.
I quickly hacked and bodged my way into a solution and then over the last few weeks I put some more effort into making a usable tool out of my quick hacking so if any one else encounters this problem there would be one more option to try and fixed it.
You can find it here fit-file . And you can check out the code on github.
The tool is purely browser based. It’s just a simple static web page written in vanila javascript.
Right now it’s basic and implements only a subset of the fit protocol which is needed to work with Zwift fit files. My plans are over time to develop this into a more general tool for fit file debugging and analyses.
Some technical parts
Let’s see why this file is broken in more detail.
FIT files are binary and that’s a bit hard to work with (if you are human, at least).
Running FitCSVTool.jar from the FIT SDK with debug option gives the following output:
$ java -jar ./FitCSVTool.jar -d ./2021-06-13-04-59-38.fit
FIT CSV Tool - Protocol 2.0 Profile 21.40 Release
Fit.Decode: Starting decode...
Running FIT verification tests...
Fit.Decode: 0x0C - FILE_HDR
Fit.Decode: 0x10 - FILE_HDR
Fit.Decode: 0x64 - FILE_HDR
Fit.Decode: 0x00 - FILE_HDR
Fit.Decode: 0x00 - FILE_HDR
Fit.Decode: 0x00 - FILE_HDR
Fit.Decode: 0x00 - FILE_HDR
Fit.Decode: 0x00 - FILE_HDR
Fit.Decode: 0x2E - FILE_HDR
Fit.Decode: 0x46 - FILE_HDR
Fit.Decode: 0x49 - FILE_HDR
Fit.Decode: 0x54 - FILE_HDR
Exception in thread "main" com.garmin.fit.FitRuntimeException: FIT decode error: File Size is 0. Error at byte: 7
According to the FIT file spec bytes 4 to 7 are part of the file header where the length of all Data Records needs to be written.
Doing that allows the check to continue, at least until it finds that something is missing at the end of the file. My first conjecture was that it’s simply missing a CRC, but adding it still resulted in an error.
$ java -jar ./FitCSVTool.jar -d ./2021-06-13-04-59-38.fit
...
Fit.Decode: 0x3F - FIELD_DATA
Fit.Decode: 0xFF - FIELD_DATA
Fit.Decode: 0x7F - FIELD_DATA
Exception in thread "main" com.garmin.fit.FitRuntimeException: FIT decode error: Unexpected end of input stream at byte: 434326
at com.garmin.fit.Decode.resume(Decode.java:434)
at com.garmin.fit.Decode.read(Decode.java:348)
at com.garmin.fit.Decode.read(Decode.java:343)
at com.garmin.fit.MesgBroadcaster.run(MesgBroadcaster.java:314)
at com.garmin.fit.test.Tests.run(Tests.java:54)
at com.garmin.fit.csv.CSVTool.run(CSVTool.java:187)
at com.garmin.fit.csv.CSVTool.main(CSVTool.java:332)
After a while I noticed that the last message is only about half the length of the previous one, while both being of the same type, data RECORD message. Good, now removing the last data record and adding the CRC allows the file to validate and most important of all finally be converted to a text csv format.
$ java -jar ./FitCSVTool.jar -d ./2021-06-13-04-59-38.fit
Fit.Decode: 0x03 - RECORD
Fit.Decode: 0x0F - FIELD_DATA
...
Fit.Decode: 0x7F - FIELD_DATA
Fit.Decode: Expecting next 2 bytes to be end of file CRC = 0xF6AB
Fit.Decode: 0xAB - RECORD
Fit.Decode: 0xF6 - RECORD
FIT binary file ./fixed-2021-06-13-04-59-38.fit decoded to ./fixed-2021-06-13-04-59-38*.csv files.
Opening that file we can see it ends at row 11741 with a Data Record message. While this validates as a correct FIT file it does not cover the minimum requirements for being a valid FIT Activity file and thus can not be uploaded to Garmin Connect. It needs to be appended with Stop Event message, and at least one Lap, Session and Activity messages.
Now we simply need to write a parser in order to extract all the information we need from the binary file and after that write an encoder that constructs those messages along with their message definitions and encodes the whole thing back into FIT binary format 
TIP: fire up the developer console (F12) in your browser and you can see the fit file in JSON format.
