To the contrary, I have quite a bit of experience in light stuff and chop.
I usually sail light (about 123kg) and now with my daughter 105, but I have also raced in the 160 range with no difference in finish position relative to normal.
If there is chop or especially in gusty conditions, the more weight the faster and harder you can drive the boat. Once you both hit the trap and 1 team has to depower, light weight does not mean squat. The light team has to work twice as hard to keep pace. This is downwind too, the boat is a lot better behaved and easier to drive with added weight.

My point was even with swings of 60 kg we are talking a “potential” of at most a second or 2. When I win in light stuff or lose in big stuff it is because I got beat, not because of my weight. Sailing is a big mental thing and light stuff especially. I would be willing to bet the heavy team who got their butt kicked in the light stuff was sitting still and bitching the whole time about being too heavy, where the light team was moving around and driving the boat.