Currently in the campaign there is a 'gunslinging' warlock who has a feat from homebrew book and another homebrew magic weapon. The magic item allows him to add any Eldritch Blast related invocations to the weapon. In this case the Warlock has the Agonizing Blast invocation to add their Charisma modifier to damage. On top of this they have a feat from a book that states: 
Six-Gun Ace

You have perfected the art of shooting with your revolver or hand cannon. Increase your Dexterity score by 1, to a maximum of 20. You get proficiency in revolvers and hand cannons if you don't have it already, as well as the following benefits:

• You cannot be disarmed while wielding a revolver or hand cannon in one hand and no weapons in your other hand.
• When you make an attack with your revolver or hand cannon and are not holding anything in your other hand, you can use your bonus action to make two additional shots. You do not add your ability score modifier to the damage of these attacks, unless that modifier is negative

If the warlock was to use this feat would they lose the benefits of Agonizing Blast because the damage agonizing blast provides is an ability score modifier. 

I would have to rule that is true.   The specific over-rides the general.  The feat is a specific rule allowing multiple shots at the cost of the ability score modifier only when using this feat.     The other stuff is a general rule for general use.

To me it seems that the feat was attempting to model two weapon fighting. And it’s referring to the ability modified to damage that is inherent in attacks. And is not referring to any possible other modifier. 

This exact situation could occur in a game using only the base rules. (Although it would be unlikely) A level 12+ pact of the blade warlock could make an offhand bonus attack with their pact weapon, after making an attack with a different weapon (both light). If the warlock had lifedrinker he could add his charisma to the off hand damage (as necrotic). 

As a dm I would allow the invocation effect to take place. 

I will caveat it that this is my RAW interpretation of the rules though and certainly wouldn’t fault a dm for adjusting some homebrew features that are getting out of hand. 

The feat is worded in a very specific way, and the attacks originate from the bonus action it grants. In addition, it's still quite good even without the "no modifiers" ruling. So I would rule the feat overrides the weapon in this case.