As stated above, the text of Counterspell and the rule in Xanathar's Guide make it clear that a spell must have components to be able to target the caster of the spell with Counterspell.

https://www.dndbeyond.com/spells/2051-counterspell

CASTING TIME

1 Reaction *

 

* - which you take when you see a creature within 60 feet of you casting a spell

https://www.dndbeyond.com/sources/xgte/dungeon-masters-tools#PerceivingaCasteratWork

But what about the act of casting a spell? Is it possible for someone to perceive that a spell is being cast in their presence? To be perceptible, the casting of a spell must involve a verbal, somatic, or material component. The form of a material component doesn’t matter for the purposes of perception, whether it’s an object specified in the spell’s description, a component pouch, or a spellcasting focus.

If the need for a spell’s components has been removed by a special ability, such as the sorcerer’s Subtle Spell feature or the Innate Spellcasting trait possessed by many creatures, the casting of the spell is imperceptible. If an imperceptible casting produces a perceptible effect, it’s normally impossible to determine who cast the spell in the absence of other evidence.

The question has been answered. There is no need to drag the debate on any further. As such I am locking this thread.