So I'm making a character for an upcoming campaign that's a Goliath Fighter (Rune Knight) that also has a dip into Spirits Bard, but that's beside the point. Basically, there are three things that this dude is going to do in combat: get big with Giant's might and grapple stuff (advantage on str checks AND double proficiency in Athletics from Bard, and I plan on taking Tavern Brawler and Grappler), invoke runes, and use my insane carrying capacity (Powerful Build from Goliath and being Large means that with a Str of 20 I'll have a max lifting capacity of 2400 pounds) to grab big things and- I don't know- plop them onto enemies to restrain them and do some bludgeoning damage? I'm not really sure how this would work, because RAW, improvised weapons are supposed to damage equal to a weapon they resemble, so if I picked up a 40-foot tree (average weight around 1000-1200 pounds according to this Quora article), one could say that it resembles a greatclub and thus would do 2d6 bludgeoning, but I feel like smacking something with a tree like Kratos does in this video would have to do a LOT more than average 7 damage to a target. However, I also get that it would be pretty game-breaking if I could just bonk things with trees wherever I go, so I don't know where a balance might be found here. What are your thoughts on how much damage stuff like this should do, and how can it be balanced so that I can't just smack things with telephone poles wherever I go?

Improvised weapons haven't dealt damage based on their weight since 3.5. 

Also note that being able to carry/drag insanely heavy objects does not also automatically allow you to attack with those objects.

The "how do you balance this" question is something I'd advice you to take with your DM ahead of the campaign but I wouldn't expect (or demand) too much tbh. The weapons in the equipment/magic item lists are balanced for the game and they max out at 1d12/2d6 for normal ones and +1-3 and possibly an extra dice for higher level magic ones.

The issue is that 5e is not balanced around size. You could make an argument that a improvised weapon should deal damage based on in increased weapon size optional rule (DMG 278). I have a Rune Knight in a campaign I am currently running that I allow to work that way, with a few caveats: 

1. The attack has to be at disadvantage
2. The improvised weapon breaks after a hit or miss

But ultimately that's something you have to work out with your DM.    

The increased weapon size rule is exclusively for monsters, which is why it's in the DMG rather than the PHB and why it's buried deep in the chapter called "Creating a Monster."

I'd also argue that a tree is nothing like a greatclub. It's got all these branches and leaves on it. You would deal 1d4 damage for an improved weapon plus the bonus from Giant's Might once a turn. Is that disappointing? Probably, but getting bigger in 5e isn't supposed to be about dealing amazing damage - they have intentionally shut that down.  It's about increasing your area of control, making better skill checks, and being able to grapple/shove creatures that you otherwise couldn't.

Well, the DMG rules do include the situation of wielding off size weapons that would only likely apply to player characters, you would build the attack issues into a monsters stat block. Like I said, its an option.

The problem with increased size is that it over values things like PAM and Sentinel, at the expense of direct damage. I'm a big proponent of working with your DM on the character concept, and using forums to see the mechanical outcome of potential house rules. In my case I allow improvised larger weapons, but not manufactured ones. I like it to be a one-off encounter specific thing, not a habit.

Also, I feel martial classes can get a bit of the shaft in 5e, but that may just be my bias.