I'm running a campaign within the Forgotten Realms (pre-spellplague) and am planning on sending my players to Avernus for a session or two. Iirc Mystra's Weave only extends across the Material Plane and a God's influence is weaker in the Outer Planes (excluding their home plane ofc) so I'm presuming there'd be some-sort of negative effect but I can't find anything concrete from searching around.

The party has a Sorcerer, a Bard and a Cleric of Selune, the Cleric I'm guessing would have difficulty recovering Spellslots but what about the other Spellcasters? I know Spellcasting works in the Outer Planes but I don't know the specifics. Does Asmodeus have his own equivalent to the Weave for them to draw from? Do their spell malfunction like during the Time of Troubles or do they just somehow work normally?

The default is everything just works normally. No fuss, no muss. 
Of course, if you want to complicate things you are free to do so. If I were going to, I’d make sure to let the players know how things will work before they go, do they can prepare their characters accordingly, or at least know what they’re getting into. 

i like the idea of azmodeus having his own weve. maybe the screams of the dammed are audible when you cast a spell, or you have advantage on rolling for fire damage

Mystra's Weave is not just limited to the Material Plane, although one could argue that the Shadowfell has closer ties to the Shadow Weave, the Weave is still available on ALL planes.

As for Divine Casters, it would be appropriate to limit certain abilities or spells (at your discretion).

If you really want rules for how different planes alter spells, you could try looking for the old Planescape Box Set.  The 2nd Edition one.  It did have rules for how spellcasters were affected by being on different planes.