There doesn't seem to be any official rules on how to use Passive skills in a game so I did a lot of reading and thinking about how they could be used and this is what I've come up with.
When players are talking to an NPC or exploring a dungeon there are of course things they could find through making a check with one of these 3 skills. When I create the dungeon I will set DCs for some of these things. Usually a player says they are looking for something specific and I ask them for a check but I will also look at each players passive skills. If their passive skill meets the DC of the check I will ask the player to roll. I've seen people saying the passive skill is the floor for making any check and you cannot roll lower than that but that is way too powerful and nearly negates the 11th level rogue feature.
My way of asking the player to make a roll when their passive meets the DC feels right to me. It's like using your peripheral vision. You think you see movement over to the side so you turn your head and look there (roll a perception check). You may still fail to see the rogue hiding in the bushes but you thought you saw something. I might even extend this for other checks like nature, religion where the PC is trying to discover something.
TLDR: When a players passive skill score meets the DC of a check I ask them for a role without them having to tell me they are trying to discover something.
However, passive skills are for tasks done repeatedly representing an average result or when the DM does not want the players to roll dice. Passive does not refer to the character being passive it refers to the player being passive because they are not rolling dice.
This means that if a character is not taking some sort of action or doing something related to the skill in the context of an encounter then the passive check doesn't apply.
Perception tends to apply all the time because characters are always considered to be aware and alert. However, the travel rules explicitly state that a character doing mapping or focusing on some other task does NOT use their passive perception when noticing things around them. Passive skills require characters to be taking a relevant action.
Some examples:
The characters are in a room in a dungeon containing a desk. A character decides to investigate the desk. Often the DM might ask for an investigation roll. Let's say it fails, so the player investigates again, and again, and again, and again. In situations, where there is no consequence for failure and no time limit then there is no reason a character can't repeat an action. The only cost is time. In this case, for an action done repeatedly the DM could use the passive investigation score and then narrate that after 10 minutes of searching and examining the character finally finds something or not. The DM could ask the character/player how long they plan to search and if this is greater than whatever time the DM decides would constitute a task done repeatedly then the DM could use the passive score.
Passive scores do NOT invalidate the Reliable Talent feature. Rogues get a minimum of 10 on their proficient skill checks whether made under pressure, in the midst of a death defying situation, or when they only get one chance. Passive scores do NOT apply to any of those scenarios.
A passive score is only a "floor" when it is a task done repeatedly (or when the DM doesn't want the players rolling dice) - repeating a task requires time and a lack of consequences for not succeeding.
Another example:
A wizard performing research at a library could use an arcana check to see if they find what they need in the first day. If the check fails then if the wizard's passive arcana skill is high enough then the DM would just narrate "After a week of searching through the stacks of the dusty library you finally find the parchment describing the information you are looking for". Passive scores are useful in situations where success is just a matter of time.
Why does passive perception get used so much? Especially in a fight where each action is 6 seconds and it is not being repeated? Because the DM doesn't want to waste time with everyone rolling dice. It is much faster and easier to just check the passive score against the one stealth roll to see whether the target is noticed.
Repeated tasks representing an average result and situations where the DM doesn't want to have the players roll dice.
Last example:
The party listens to a 30 minute speech by the candidate for mayor of the town. He goes over policies, what he wants to do, that he will crack down on crime. Do you have the players roll insight every minute to see what they believe? Do you assume the players just believe everything he is saying is the truth? Nope - this is a case where you could apply passive insight. Characters with high passive insight might notice when he is telling the truth, when he is bending it and when he is outright lying and the DM would just narrate the results. The DM would let character A know that they think the candidate wasn't being completely truthful about cracking down on crime for example.
Anyway :) ... that is how I tend to use passive skills (which I think is both consistent with RAW and makes sense).
There doesn't seem to be any official rules on how to use Passive skills in a game
Here are the core mechanics for a passive check. One of the big takeaways here is that they should represent "the average result for a task done repeatedly", although they can clearly be used for other things, as covered in the Stealth rules; despite how the rules for grappling work, the general way WOTC seems to intend 5E gameplay to work is that when you make a skillcheck that contests someone else's skillcheck but they haven't made theirs because they can't due to action economy and the nature of turns, the person whose turn it isn't should have their passive check set the DC for the active turn participant. This is e.g. how spells that are resisted by Str/Dex/Wis checks work (with a functional -2 modifier to the spell's passive check).
While we don't have direct rules for Insight and Investigation like we do Perception, it's relatively easy to port the above rules over:
The base Deception DC for a lie you tell can be your target's passive Insight, with the DM applying advantage or disadvantage for particularly plausible or implausible lies (in the same way visibility conditions modify your chance to hide visually), and you'll have simulated the stealth rules above for social stealth. Similarly, a target's passive Insight can set your Disguise Kit DC.
If you like, you could use similar logic to have passive Investigation set the DC for Forgeries.
Not limited to just Passive Investigation, more or less all of the Xanathar's rules for downtime activities are the average result of a task done repeatedly, so you'd be will within reason to make them passive checks, not active ones. Research would therefore use Passive Investigation, just as Carousing would use Passive Persuasion. This would amount to porting over the rules for noticing threats while traveling.
There doesn't seem to be any official rules on how to use Passive skills in a game so I did a lot of reading and thinking about how they could be used and this is what I've come up with.
When players are talking to an NPC or exploring a dungeon there are of course things they could find through making a check with one of these 3 skills. When I create the dungeon I will set DCs for some of these things. Usually a player says they are looking for something specific and I ask them for a check but I will also look at each players passive skills. If their passive skill meets the DC of the check I will ask the player to roll. I've seen people saying the passive skill is the floor for making any check and you cannot roll lower than that but that is way too powerful and nearly negates the 11th level rogue feature.
My way of asking the player to make a roll when their passive meets the DC feels right to me. It's like using your peripheral vision. You think you see movement over to the side so you turn your head and look there (roll a perception check). You may still fail to see the rogue hiding in the bushes but you thought you saw something. I might even extend this for other checks like nature, religion where the PC is trying to discover something.
TLDR: When a players passive skill score meets the DC of a check I ask them for a role without them having to tell me they are trying to discover something.
How does everyone else use these passive scores?
Its up to you how you want to use them.
However, passive skills are for tasks done repeatedly representing an average result or when the DM does not want the players to roll dice. Passive does not refer to the character being passive it refers to the player being passive because they are not rolling dice.
This means that if a character is not taking some sort of action or doing something related to the skill in the context of an encounter then the passive check doesn't apply.
Perception tends to apply all the time because characters are always considered to be aware and alert. However, the travel rules explicitly state that a character doing mapping or focusing on some other task does NOT use their passive perception when noticing things around them. Passive skills require characters to be taking a relevant action.
Some examples:
The characters are in a room in a dungeon containing a desk. A character decides to investigate the desk. Often the DM might ask for an investigation roll. Let's say it fails, so the player investigates again, and again, and again, and again. In situations, where there is no consequence for failure and no time limit then there is no reason a character can't repeat an action. The only cost is time. In this case, for an action done repeatedly the DM could use the passive investigation score and then narrate that after 10 minutes of searching and examining the character finally finds something or not. The DM could ask the character/player how long they plan to search and if this is greater than whatever time the DM decides would constitute a task done repeatedly then the DM could use the passive score.
Passive scores do NOT invalidate the Reliable Talent feature. Rogues get a minimum of 10 on their proficient skill checks whether made under pressure, in the midst of a death defying situation, or when they only get one chance. Passive scores do NOT apply to any of those scenarios.
A passive score is only a "floor" when it is a task done repeatedly (or when the DM doesn't want the players rolling dice) - repeating a task requires time and a lack of consequences for not succeeding.
Another example:
A wizard performing research at a library could use an arcana check to see if they find what they need in the first day. If the check fails then if the wizard's passive arcana skill is high enough then the DM would just narrate "After a week of searching through the stacks of the dusty library you finally find the parchment describing the information you are looking for". Passive scores are useful in situations where success is just a matter of time.
Why does passive perception get used so much? Especially in a fight where each action is 6 seconds and it is not being repeated? Because the DM doesn't want to waste time with everyone rolling dice. It is much faster and easier to just check the passive score against the one stealth roll to see whether the target is noticed.
Repeated tasks representing an average result and situations where the DM doesn't want to have the players roll dice.
Last example:
The party listens to a 30 minute speech by the candidate for mayor of the town. He goes over policies, what he wants to do, that he will crack down on crime. Do you have the players roll insight every minute to see what they believe? Do you assume the players just believe everything he is saying is the truth? Nope - this is a case where you could apply passive insight. Characters with high passive insight might notice when he is telling the truth, when he is bending it and when he is outright lying and the DM would just narrate the results. The DM would let character A know that they think the candidate wasn't being completely truthful about cracking down on crime for example.
Anyway :) ... that is how I tend to use passive skills (which I think is both consistent with RAW and makes sense).
Here are the core mechanics for a passive check. One of the big takeaways here is that they should represent "the average result for a task done repeatedly", although they can clearly be used for other things, as covered in the Stealth rules; despite how the rules for grappling work, the general way WOTC seems to intend 5E gameplay to work is that when you make a skillcheck that contests someone else's skillcheck but they haven't made theirs because they can't due to action economy and the nature of turns, the person whose turn it isn't should have their passive check set the DC for the active turn participant. This is e.g. how spells that are resisted by Str/Dex/Wis checks work (with a functional -2 modifier to the spell's passive check).
Here are the rules for a Stealth check being contested by Passive Perception. Here are the rules for contested checks.
Here are the rules for applying Passive Perception to noticing threats while traveling.
While we don't have direct rules for Insight and Investigation like we do Perception, it's relatively easy to port the above rules over: