I’m modelling this question for a normal adult dragon, though it could be interesting to know the same answer for an ancient/ young dragon if there is any good info out there.

assuming the dragon was willing, how many medium creatures could it carry on its back, while still allowing it to fly? I’ve looked around and there don’t seem to be many rules or guidelines concerning this question, so if anybody has a good answer, then I’d love to know about it. 

By the numbers, an adult red dragon has a STR of 27, so multiply that by 15, then quadruple it because it is a huge creature and you get a carrying capacity of 1620lbs. But personally, I think an adult red dragon could fly off with an elephant. So at that point, it's really just a question of how many people you want to try to stack on that adult red dragon (and of course how it feels about the matter).

I’d suggest that the answer is, what makes your game the most fun? 

Based on those guesses it’s about 12 max. But at the end of the day, if there are no rules on it, I’ll happily let the rule of cool take over. Thanks for the reply!

This is where the rule of cool really comes into play. For my game specifically I’m not sure yet, and there’s lots of stuff in my campaign I have to work out. But I’d like to think a good sized dragon could probably carry a few dozen people give or take. Thanks for the reply!

Keep in mind. If you don’t make a specific ruling, you can adjust how many people will fit based on the needs of the scenario. For one situation everyone can fit on the dragon, but for another situation with a different dragon it might take 2 or 3 trips to transport everyone. Splitting the party can be fun when it’s done right. 

I imagine it would be ridiculously difficult to hold on as well. 

A few could fit in the mouth as well

The correct answer is "as many as it will allow" but the real answer is "up to the DM, unless you want to get bogged down in math"