Not all offensive uses of spells are obvious. Some are even intended to be used outside of combat (Charm Person, for example).
Let's consider a sorceror with subtle spell. You're in a crowded bar. You cast subtle phantasmal force on someone. Do his friends realize he's been attacked? It just looks like a psychotic episode. Does the target even realize he's been attacked? (Aside from potentially by whatever his phantom fear is). Should initiative even be rolled? What if he passes his saving throw, instead. He doesn't even know he was targeted by a spell, right?
This is 100% DM territory. All people can tell is how they'd personally handle it. I'll give how I would, but there's no official answer.
Phantasmal Force: In a world where magic is common enough, I'd definitely start initiative. People might start getting accosted and threatened if the culprit doesn't come forward and it would likely be a chaotic mess. That is... If they freak out. Phantasmal Force doesn't actually deal damage unless you make it something hazardous. So you could just make it a creature or object that disturbs them but isn't actively harmful... That's less likely to cause suspicion, unless it happens to multiple people or comes completely out of the blue. So such a situation would be determined by just what the effect is and the circumstances around the use. (Phantasmal Killer would likely be the first result here, as it definitely deals damage consistently and the person would be more distressed.)
Dominate Person: I'd only call for initiative in this situation if this was used to start a fight. If used in a way that was subtle enough to mask the behavior, I'd say nothing happens immediately. And if the players had studied their target and were having it act within the bounds of their normal activity while around people who knew them (and not being overly suspicious around strangers) there would be no checks moving forward. However, if they had the person acting suspiciously (which is a lower bar around people who know them) there would be Insight checks from NPCs to determine what's wrong, and possible investigation.
[...] What if he passes his saving throw, instead. He doesn't even know he was targeted by a spell, right? [...]
I remember similar threads about a spell being noticeable (*). There are some rules in the PHB ("Awareness of Being Targeted") and in XGtE ("Perceiving a Caster at Work"). Also Do you always know when you’re under the effect of a spell?
So based on those rules, I'd say that when using something similar to Sorcerer's Subtle Spell or the Psychic Spells feature, if the creature succeeds on the Saving Throw and there are no secondary effects after succeeding, it won't know it was targeted by spells such as Charm Person, Dominate Person, Phantasmal Force, or Suggestion.
Regarding your main question, I think, as Athanar90 said, it's DM territory and IMO it would depend on how the spells are used, whether they deal damage, etc.
Not all offensive uses of spells are obvious. Some are even intended to be used outside of combat (Charm Person, for example).
Let's consider a sorceror with subtle spell. You're in a crowded bar. You cast subtle phantasmal force on someone. Do his friends realize he's been attacked? It just looks like a psychotic episode. Does the target even realize he's been attacked? (Aside from potentially by whatever his phantom fear is). Should initiative even be rolled? What if he passes his saving throw, instead. He doesn't even know he was targeted by a spell, right?
What about subtle Dominate Person?
When combat starts between two or more creatures, every participant rolls Initiative to determine the order of turns.
If a combatant is surprised by combat starting, that combatant has Disadvantage on their Initiative roll. For example, if an ambusher starts combat while hidden from a foe who is unaware that combat is starting, that foe is surprised.
As DM i'd give Advvantage to ambusher's Initiative, Disadvvantage to surprised creature and use Initiative scores instead of rolling Initiative for following result;
Now in very rare case where the surprised creature in combat still doesn't know who are the other participants or what was attempted when it's turn comes up, as DM i can say it an intuition of imminent danger is felt despite not noticing any result from the spell.
Failed Dominate Person described as the creature about to drink briefly losing control of its arm holding the mug inexplicably, or resisting voice in its head urging to do something to me is a freakin way to start combat 😊
Ask the players how they would like it to be handled at the table, letting them know it goes both ways. Personally as a player and DM id like some things to not start combat as yeah people don't know something is up.
But I can understand if the players would think it is unfair they did not get turns while things were going down while they were unaware of it. In which case when the players cast dominate or whatever subtly the opposition gets a feeling something is off and roll init as they are on guard. Sort of like in video games where the guards spot enough to look for things, but they haven't fully seen you their awareness meter is at half. they start looking around, trying to find out what is up. but they are not charging at you swords drawn.
The Dungeon Master Guide offer more guidance on rolling Initiative;
Rolling Initiative
Combat starts when—and only when—you say it does. Some characters have abilities that trigger on an Initiative roll; you, not the players, decide if and when Initiative is rolled. A high-level Barbarian can't just punch their Paladin friend and roll Initiative to regain expended uses of Rage.
In any situation where a character's actions initiate combat, you can give the acting character Advantage on their Initiative roll. For example, if a conversation with an NPC is cut short because the Sorcerer is convinced that NPC is a doppelganger and targets it with a Chromatic Orb spell, everyone rolls Initiative, and the Sorcerer does so with Advantage. If the doppelganger rolls well, it might still act before the Sorcerer's spell goes off, reflecting the monster's ability to anticipate the spell.
Now in very rare case where the surprised creature in combat still doesn't know who are the other participants or what was attempted when it's turn comes up, as DM i can say it an intuition of imminent danger is felt despite not noticing any result from the spell.
Failed Dominate Person described as the creature about to drink briefly losing control of its arm holding the mug inexplicably, or resisting voice in its head urging to do something to me is a freakin way to start combat 😊
A creature only knows they were the target of the spell if the spell does something obvious. A failed dominate person doesn't do anything obvious, so you wouldn't know you were the target of a spell, RAW. ("Unless a spell has a perceptible effect, a creature might not know it was targeted by a spell at all. An effect like crackling lightning is obvious, but a more subtle effect, such as an attempt to read a creature’s thoughts, typically goes unnoticed, unless a spell says otherwise.")
Indeed, even a successful dominate person doesn't let the target know they were charmed (unlike charm person - the only reason someone knows they were the target of a charm person is because of this sentence: "When the spell ends, the target knows it was Charmed by you."). Maybe using the telepathic link lets it know it was the target of a spell (but which spell? Several spells allow telepathy, as do several racial and subclass features.) So even a successful dominated person would be unaware they were the target of a spell, especially if you didn't bother with the telepathic commands.
RAW PC don't know monsters resist any damage dealt though i still like to describe attack not damaging as much as it should for example. 😊
With Initiative, what matter here is that combatant knows its surprised by combat starting despite not perceiving any active threat, even if it only comes down to intuition like i said.
RAW PC don't know monsters resist any damage dealt though i still like to describe attack not damaging as much as it should for example. 😊
With Initiative, what matter here is that combatant knows its surprised by combat starting despite not perceiving any active threat, even if it only comes down to intuition like i said.
The surprise rules are dumb. They only make sense in things like the example they give (where everyone is aware of everyone else, just not expecting hostilities, and then the sorceror starts actively and visibly casting a spell. (Or someone draws a sword in the middle of negotiations, or other similar situations).
--They don't make sense when one side is entirely unaware of the other. (Which is determined by stealth vs. perception, not the initiative roll).
--They don't make sense when the hostile act is imperceptible. (ie, someone casting a subtle spell).
Because in both those cases, the result of the offensive action is what starts combat, not the taking of the offensive action.
And the rules are quite specific - you have no idea you were the target of the spell unless the spell has some obvious perceptible phenomenon (or explicitly tells you that you know, like Charm Person). If you have no idea, that means there's no spidey sense about it. You are completely unaware the spell was cast at you.
Resistance has no such rule. It seems quite sensible to describe the attack in ways that are less effective than it should have been, or the wound closing rapidly, or something, because the effect of the wound would be an obviously perceptible phenomenon. And, well, the PCs are certainly entitled to know some things about monster health, like when the monster is bloodied (otherwise the new Rogue sub-class wouldn't be able to trigger its teleport-attack). That suggests that while the PCs might not get to know a monster's exact hp total, they know in general terms how its faring health-wise.
Your reading of the rules would make Friends unusable, because you'd have it start combat (and thus auto-fail). That seems like a bad rules take. (Not to mention the several enchantment spells that give advantage on the save if it's in combat with you - meaning it can be cast while not in combat, and thus without starting combat -- if it started combat, with the current Surprise rules it would always be actually cast during combat).
Not all offensive uses of spells are obvious. Some are even intended to be used outside of combat (Charm Person, for example).
Let's consider a sorceror with subtle spell. You're in a crowded bar. You cast subtle phantasmal force on someone. Do his friends realize he's been attacked? It just looks like a psychotic episode. Does the target even realize he's been attacked? (Aside from potentially by whatever his phantom fear is). Should initiative even be rolled? What if he passes his saving throw, instead. He doesn't even know he was targeted by a spell, right?
What about subtle Dominate Person?
This is 100% DM territory. All people can tell is how they'd personally handle it. I'll give how I would, but there's no official answer.
Phantasmal Force: In a world where magic is common enough, I'd definitely start initiative. People might start getting accosted and threatened if the culprit doesn't come forward and it would likely be a chaotic mess. That is... If they freak out. Phantasmal Force doesn't actually deal damage unless you make it something hazardous. So you could just make it a creature or object that disturbs them but isn't actively harmful... That's less likely to cause suspicion, unless it happens to multiple people or comes completely out of the blue. So such a situation would be determined by just what the effect is and the circumstances around the use. (Phantasmal Killer would likely be the first result here, as it definitely deals damage consistently and the person would be more distressed.)
Dominate Person: I'd only call for initiative in this situation if this was used to start a fight. If used in a way that was subtle enough to mask the behavior, I'd say nothing happens immediately. And if the players had studied their target and were having it act within the bounds of their normal activity while around people who knew them (and not being overly suspicious around strangers) there would be no checks moving forward. However, if they had the person acting suspiciously (which is a lower bar around people who know them) there would be Insight checks from NPCs to determine what's wrong, and possible investigation.
I remember similar threads about a spell being noticeable (*). There are some rules in the PHB ("Awareness of Being Targeted") and in XGtE ("Perceiving a Caster at Work"). Also Do you always know when you’re under the effect of a spell?
So based on those rules, I'd say that when using something similar to Sorcerer's Subtle Spell or the Psychic Spells feature, if the creature succeeds on the Saving Throw and there are no secondary effects after succeeding, it won't know it was targeted by spells such as Charm Person, Dominate Person, Phantasmal Force, or Suggestion.
Regarding your main question, I think, as Athanar90 said, it's DM territory and IMO it would depend on how the spells are used, whether they deal damage, etc.
(*) this is one of them: 2024 Warlock 'Psychic Spells' Feature
When combat starts between two or more creatures, every participant rolls Initiative to determine the order of turns.
If a combatant is surprised by combat starting, that combatant has Disadvantage on their Initiative roll. For example, if an ambusher starts combat while hidden from a foe who is unaware that combat is starting, that foe is surprised.
As DM i'd give Advvantage to ambusher's Initiative, Disadvvantage to surprised creature and use Initiative scores instead of rolling Initiative for following result;
Ambusher 15 + DEX mod
Surprised 5 + DEX mod
Now in very rare case where the surprised creature in combat still doesn't know who are the other participants or what was attempted when it's turn comes up, as DM i can say it an intuition of imminent danger is felt despite not noticing any result from the spell.
Failed Dominate Person described as the creature about to drink briefly losing control of its arm holding the mug inexplicably, or resisting voice in its head urging to do something to me is a freakin way to start combat 😊
Ask the players how they would like it to be handled at the table, letting them know it goes both ways. Personally as a player and DM id like some things to not start combat as yeah people don't know something is up.
But I can understand if the players would think it is unfair they did not get turns while things were going down while they were unaware of it. In which case when the players cast dominate or whatever subtly the opposition gets a feeling something is off and roll init as they are on guard. Sort of like in video games where the guards spot enough to look for things, but they haven't fully seen you their awareness meter is at half. they start looking around, trying to find out what is up. but they are not charging at you swords drawn.
The Dungeon Master Guide offer more guidance on rolling Initiative;
A creature only knows they were the target of the spell if the spell does something obvious. A failed dominate person doesn't do anything obvious, so you wouldn't know you were the target of a spell, RAW. ("Unless a spell has a perceptible effect, a creature might not know it was targeted by a spell at all. An effect like crackling lightning is obvious, but a more subtle effect, such as an attempt to read a creature’s thoughts, typically goes unnoticed, unless a spell says otherwise.")
Indeed, even a successful dominate person doesn't let the target know they were charmed (unlike charm person - the only reason someone knows they were the target of a charm person is because of this sentence: "When the spell ends, the target knows it was Charmed by you."). Maybe using the telepathic link lets it know it was the target of a spell (but which spell? Several spells allow telepathy, as do several racial and subclass features.) So even a successful dominated person would be unaware they were the target of a spell, especially if you didn't bother with the telepathic commands.
RAW PC don't know monsters resist any damage dealt though i still like to describe attack not damaging as much as it should for example. 😊
With Initiative, what matter here is that combatant knows its surprised by combat starting despite not perceiving any active threat, even if it only comes down to intuition like i said.
The surprise rules are dumb. They only make sense in things like the example they give (where everyone is aware of everyone else, just not expecting hostilities, and then the sorceror starts actively and visibly casting a spell. (Or someone draws a sword in the middle of negotiations, or other similar situations).
--They don't make sense when one side is entirely unaware of the other. (Which is determined by stealth vs. perception, not the initiative roll).
--They don't make sense when the hostile act is imperceptible. (ie, someone casting a subtle spell).
Because in both those cases, the result of the offensive action is what starts combat, not the taking of the offensive action.
And the rules are quite specific - you have no idea you were the target of the spell unless the spell has some obvious perceptible phenomenon (or explicitly tells you that you know, like Charm Person). If you have no idea, that means there's no spidey sense about it. You are completely unaware the spell was cast at you.
Resistance has no such rule. It seems quite sensible to describe the attack in ways that are less effective than it should have been, or the wound closing rapidly, or something, because the effect of the wound would be an obviously perceptible phenomenon. And, well, the PCs are certainly entitled to know some things about monster health, like when the monster is bloodied (otherwise the new Rogue sub-class wouldn't be able to trigger its teleport-attack). That suggests that while the PCs might not get to know a monster's exact hp total, they know in general terms how its faring health-wise.
Your reading of the rules would make Friends unusable, because you'd have it start combat (and thus auto-fail). That seems like a bad rules take. (Not to mention the several enchantment spells that give advantage on the save if it's in combat with you - meaning it can be cast while not in combat, and thus without starting combat -- if it started combat, with the current Surprise rules it would always be actually cast during combat).
RAW Surprise was designed for when creature is unaware combat is starting as cited in the example for it.
When combat starts is entirely up to DM, spellcasting may very well do so and others may react etc