its more about inertia on the descents
Ok. Iām starting to be convinced that I may have erred in my understanding.
An experiment you might try āat homeā would be to find a descent of reasonable length and grade, preferably straight or with no sharp corners, and coast down it with, and without, two full water bottles. My experience has been that descending with the full bottles results in a higher velocity.
You could also try this with, and without, an anvil, but it is pretty hard to hold into the anvil.
Interesting discussion given that my youngest daughters just participated in a pinewood derby race this past weekend. Pinewood derby races are an excellent example of how heavier objects roll downhill faster than lighter objects. The heavier objects have more momentum with which to overcome the friction from the air and ground. Thus, they descend faster and hold that momentum for longer once at the bottom of the hill. Cheers!
Galileo was absolutely correct. He wanted to prove that two bodies of different weight would fall at the same speed (and thus prove Aristotle wrong). So like a good scientist he chose an experiment that removed all factors other than weight as far as possible. In his case that was to use two cannonballs of the same size but different weights. And two cannonballs of the same size will have near identical aero drag coefficients, so with the same force (gravity) accelerating each and the same drag force trying to slow each, over a relatively short distance of 50 metres, he got his famous result.
Iām sure he knew that if he dropped a hammer and a feather things would appear different but that wasnāt what he was trying to prove and understand.
What does that mean for the freewheeling downhill cyclist. Well, lets put you on a bike with a set of pannier bags. First you roll down the mountain road with the panniers filled to the brim with feathers. Then you repeat rolling down the hill with the panniers full of lead.
In both cases:

The accelerating force on you, the bike, and your luggage comes from gravity and as Galileo proved, it is proportional to your mass. The heavier you gets a larger force but youāre heavier, so it has the same effect as the lower force on the lighter you.

The force opposing you is aero drag and its also the same in both cases because its the same you in the same aero tuck. But the drag on the lighter you has a bigger effect than that same drag force on the heavier you. Why?  As Newton showed us, F=ma or to put that another way round, your rate of deceleration is a=F/m where F is the same in both cases.
Fair play to you for accepting this.
For the next topical discussion shall we discuss the merits of the feather given the fact that being lighter makes no difference to your gravitational pull going uphill.
Would it make any difference if the bike was on a conveyor belt
Being evil and having been forced by my own mental OCD to create a spreadsheet for this, Iāve made poor old Galileo climb back up the Tower of Pisa with two cast iron cannonballs  one 12lbs and the other drilled so its the same size but only 10lbs.
When he simultaneously drops them, they do actually reach the ground almost 1 metre apart. So why didnāt he spot that in his original experiment (if he did it*). Well, they both would have been moving at over 58mph and would have arrived just 0.04 seconds apart. So very hard to separate visually or by the sound of impact. Especially with the equipment of era which would have been human ears, eyes and maybe the odd sandtimer.
Now if he had wanted to get a more appreciable separation, lets say a full second, he needs lugs those cannonballs up something at least 470 metres tall. Easy enough, I donāt thinks he even 500 years old yet. My preference would be the Shanghai World Finance Centre, lovely straight sides. Both cannonballs will reach their terminal velocities (76mph and 69mph respectively) before they hit the pavement, but hopefully some able assistant might detect the two separate crashes if we can find anyone brave enough to stand nearby down below.
But at least we get revenge, since now he canāt just saunter off thinking heās gifted us the idea that the force of gravity is proportional to an objectās mass and thatās all we need to know. Now heās got to account for a lot of other annoying factors he had previously designed out and left to subsequent engineers and scientists.
*Thereās a lot of debate about whether the experiment took place or not or was done by someone else. And people had proposed the idea of constant acceleration regardless of mass before Galileo and he was aware of that. Also in his defence, in the written presentation of the idea, he did mention the need for the resistance of the medium being negligible or a vacuum, so he was aware that other factors would affect falling bodies in the real world.
Dr. Charles Townsend in 1926 published an article in which he estimated the flying speed of the male deerfly at 400 yards/second. Which would be over 800 mph.
He did that, literally, by standing on a hill and observing that the deerflies around him were flying so fast āas to be a blurā. Then he just said āprobably around 400y/sec.ā
This was debunked in 1938 (that estimate, based on measuring a similarly sized object spun around on a string, was 25mph). But when I was a kid decades ago I had a ābig book of factsā that still gave the 400y/sec stat. āFactsā stick around when theyāre good ones
But what is the airspeed velocity of an unladen swallow?
African, European, Pacific, Lesser Striped, Northern RoughWinged, Mascarene, Barn?
I adapted my calculations to be for freewheeling cyclists rather than plummeting cannonballs.
With myself (weight will not be stated) alongside a 60kg racer at the top of a superlong 4% slope, before the 1st kilometre they reach their peak speed of 26mph  but Iām already faster, already 200 metres ahead, and still accelerating towards a peak of 34mph.
Going downhill weight works.