If a monk were to combine the Mage Slayer reaction attack with Stunning Strike against a mage casting a spell would it prevent the target from casting its spell if the stun succeeds?

Yes. If they can stun the caster then they are incapacitated which means they can’t take actions, which casting a spell is an action so I would say they lose the spell. 

But I could see a DM ring otherwise (the reaction attack still does damage so I believe they would require a concentration save) because the action to cast the spell was already initiated. 

That was my thought.  Would the spell casting have to be completed first before you can take your reaction?

Adjudicating Reaction Timing

Typical combatants rely on the opportunity attack and the Ready action for most of their reactions in a fight. Various spells and features give a creature more reaction options, and sometimes the timing of a reaction can be difficult to adjudicate. Use this rule of thumb: follow whatever timing is specified in the reaction’s description. For example, the opportunity attack and the shield spell are clear about the fact that they can interrupt their triggers. If a reaction has no timing specified, or the timing is unclear, the reaction occurs after its trigger finishes, as in the Ready action.

The reaction attack occurs after the trigger, so the spell has been cast already.

Quote from HeironymusZot >>

Adjudicating Reaction Timing

Typical combatants rely on the opportunity attack and the Ready action for most of their reactions in a fight. Various spells and features give a creature more reaction options, and sometimes the timing of a reaction can be difficult to adjudicate. Use this rule of thumb: follow whatever timing is specified in the reaction’s description. For example, the opportunity attack and the shield spell are clear about the fact that they can interrupt their triggers. If a reaction has no timing specified, or the timing is unclear, the reaction occurs after its trigger finishes, as in the Ready action.

The reaction attack occurs after the trigger, so the spell has been cast already.

 I think you hit the mail in the head. I was thinking it was like counterspell where it’s a reaction to seeing a spell being cast but it does say “casts a spell” so it is after completion. My bad

Yeah I think it lacks that important key word, "interrupt."  Too bad.  It would have been a cool build.

I still think Mage Slayer monk is a solid build / role, even if you cant use it to interrupt spellcasting. Being able to make 3-4 attacks every turn to proc concentration checks is an obvious plus. On top of that you have a huge speed boost that can be used to close the distance between you and the spellcaster, the ability to end charm and frighten effects with Still Mind, Evasion to avoid Fireballs and Lightning Bolts, and Diamond Soul to help avoid spells that would take you out of the fight (like Hold Person) 

Quote from Kaboom979 >>

I still think Mage Slayer monk is a solid build / role, even if you cant use it to interrupt spellcasting. Being able to make 3-4 attacks every turn to proc concentration checks is an obvious plus. On top of that you have a huge speed boost that can be used to close the distance between you and the spellcaster, the ability to end charm and frighten effects with Still Mind, Evasion to avoid Fireballs and Lightning Bolts, and Diamond Soul to help avoid spells that would take you out of the fight (like Hold Person) 

 It would be an interesting build, and I had created one in the 60+ characters that I had in DDB character builder, but never used. It is unfortunate that taking a feat instead of an ASI hurts so much on a monk that I would find it hard to actually do unless using VHuman or Custom lineage, and I could find a better feat to take instead for a niche role.

Quote from ThriKreenWarrior >>

Quote from Kaboom979 >>

I still think Mage Slayer monk is a solid build / role, even if you cant use it to interrupt spellcasting. Being able to make 3-4 attacks every turn to proc concentration checks is an obvious plus. On top of that you have a huge speed boost that can be used to close the distance between you and the spellcaster, the ability to end charm and frighten effects with Still Mind, Evasion to avoid Fireballs and Lightning Bolts, and Diamond Soul to help avoid spells that would take you out of the fight (like Hold Person) 

 It would be an interesting build, and I had created one in the 60+ characters that I had in DDB character builder, but never used. It is unfortunate that taking a feat instead of an ASI hurts so much on a monk that I would find it hard to actually do unless using VHuman or Custom lineage, and I could find a better feat to take instead for a niche role.

 I think the best subclass for it would be the Way of the Astral Self, since (i) you can deflect elemental damage and (ii) with your Astral Arms active you can make unarmed strikes with reach using Wisdom, so you could reasonably get away with not bumping your Dex to its maximum and could focus on Wisdom and one or more feats. Your AC would take a hit, but hopefully Evasion and your other defensive abilities could make up for a 1 or 2 in your AC.