So in our initial session, I threw my PCs into the sewers of a city and in there they encountered a beast (modeled on a trog, but not explicitly clear it was a trog) hunting them. Now they're level 4-5 (depending on if/when they do this side quest I'm concocting) and going back into the sewers to get rid of a nest of baddies.

My basic premise - open to ideas and input on this - is that a hydroloth or froghemoth has taken residence in some forgotten wing of the sewers and established a little kingdom over some bullywugs or trogs there. Eventually it becomes a problem that the PCs are tasked with resolving.

If going with these kinds of fodder mobs, how should I scale them up to be not too easy/not too hard threats to the PCs? I know I can boost their health or give them advantage on rolls, but if others have ideas (or homebrews for good sewer beasts) I welcome the sharing. Simply throwing more numbers at the PCs could be an initiative order nightmare I want to avoid.

The easiest way to solve this problem is use the time honored DM strategy "it doesn't matter how many HP the monster has, it dies when I feel the players have put in enough effort."

You might consider giving boss monsters special abilities to use on their turns or as mini legendary actions. Matt Colville calls this "action-oriented monsters" and it really brings unique challenges and flavor to a fight. 

Easy ways:

• More hp (as you stated)
• Higher ACs, just 1 or 2 extra AC is enough for fodder mobs
• Higher to-hit modifiers and maybe an extra attack, but keep the actual damage dice & modifier low. This makes them feel more dangerous without dealing too much damage to the party.
• Resistance to slashing, bludgeoning, or piercing damage, maybe some other damage types like poison or fire. If you go this route, only put it on the captains of those units. 

Harder:

• Strategize their encounter. How will this pack of monsters fight?
• Strategize turn by turn.
• Strategize combos between various monsters. 

When it comes to making the monsters harder to kill I tend to prefer more HP or damage resistances over higher AC most of the time. Monsters with really high AC frustrate players because they can’t land hits. Monsters with lots of HP or damage resistance but slightly lower AC lets the players feel like mighty lumberjacks/lumberjills chopping away at mighty foes. That being said, having a high AC “centerpiece” monster in the mix for them to contend with makes things feel more dynamic because then everything isn’t just samesame.

When it comes to making the monsters feel more dangerous, you got four main choices: more monsters, more  attacks, higher attack bonuses, or higher damage per hit. More monsters or attacks makes combat take longer, and high damage with a low attack modifier makes for unpredictable, swingy encounters. But higher attack bonuses on a reasonable number of attacks with okay damage per hit makes them feel dangerous (because they don’t miss), without running the risk of waffle stomping the party.

When it comes to making the monsters a tougher challenge, I like stuff that imposes conditions on the party like frightened or poisoned or restrained. It makes things harder, without making the monsters too much more dangerous if that makes sense. A handful of weenie monsters becomes a bigger challenge if half the party is poisoned for most of the fight.

Remember, multiple copies of the same creature have the same initiative, and if necessary you can use the rules for mobs.

Helpful tips! Going chonky with the HP was kinda the default, but that just might not make sense with some mobs that are expected to be trash. And long fights are slogs if not against a boss.

Higher +hit and maybe more use of conditionals sounds like a good combination that makes even the trog or bullywug more a threat without breaking their default stat blocks too much or over-chonking the trash.

The other thing you could do is make them all “minions” (all have only 1 HP), but just throw a shedload at them. The heroes can hew left and right and be mighty monster mulching machines of magnitude, the fight will still only last 4ish rounds, but they will certainly feel overwhelmed. Do it in waves and then you won’t even have to deal with a bagillion attacks every turn. Make each wave a number of minions equal to the number of attacks the party has + their average PB and send in 3 waves of them, that’ll still wrap up in about 3&½-4 rounds, give or take.  

Quote from Acuaricornio >>

If going with these kinds of fodder mobs, how should I scale them up to be not too easy/not too hard threats to the PCs?

If the party is fighting on the mob's home turf, there are all kinds of ways you can make them a bigger challenge with traps and other environmental/strategic elements in the sewers they can exploit

In other words, don't make the enemies tougher, make them smarter

Lots of good advice here, I'll echo what others have said about monsters that hit more often might feel more dangerous because they are always dealing damage. Survivability comes in the form of either not taking damage (high AC), not being as affected by damage (resistance), or being able to take more damage than your opponent (high hp). It's more fun for the players to be able to hit the monster, and adding a pile of hp is just a longer battle. Adding resistances to a type of damage, the Shield spell, or Parry (+2 or 4 AC to triggering attack; reaction) might drag a fight out some. But those are more specific and limited reductions of damage that can be overcome or depleted. 

Action Oriented Monsters are useful as the boss or boss-adjacent monster in the group, as others have pointed out. Mashing two, or more, statblocks together to create a custom creature flavored to your encounter is useful as well. The instructions for the Half-Dragon Template will work for any statblocks, but there might be a need to run through the Creating Quick Monster Stats afterword, just to make sure you didn't nudge anything too far, one way or the other.  Someone once suggested the idea of making KoboldHyena riders.

Lastly, I'll bring up the environment. A party of 5th level PC can easily handle 3 Kobolds, but put that party of PCs in quicksand that won't support more than 40lbs. and now those 3 kobolds become a distraction to the main threat of suffocation. Traps and the environment can be used in conjunction with monster attacks to complicate the encounter and make the encounter feel more deadly without altering the mob count or too much homebrew. A portcullis and a Gelatinous Cube can change outcomes if the PCs are stuck between them. Moreso, if one PC is trapped alone on the same side of the portcullis as the Cube. Amphibious monsters might use their ability to breathe water or hold their breath to drag an "Air Breather" under and let them succumb to the lack of oxygen, all the while stabbing them with their dagger or restraining them with a net. I'm told that swimming in heavy armor is quite difficult.

Swarm mechanics can be nice to represent a lot of creatures without having to keep track of individual ones. A huge-sized swarm of froghemoth spawn paints a pretty compelling picture. Just grab any swarm and scale it up, they're pretty generic.