I have a party of 4 all at 5th level. Next dungeon we're coming to is currently 2 levels, I'm thinking of adding a third.

Currently, L1 is the ruins of a castle. There are a handful of combat encounters (plus some RP moments when they free some prisoners and find one of the characters' parents dead) in L1, including (but I'm amenable to eliminating this) a half-green dragon.

L2 is a giant labyrinth with invisible walls and some other measures meant to keep a creature sealed inside (and other adventurers out). No wandering encounters any more. They just didn't make sense with the invisbile labyrinth walls.

I'm thinking of adding a full cavern underneath the labyrinth. The PCs enter on one side. Maybe it's 2-3 rooms or a single room, I haven't decided. I was thinking of finding a creature to put in there that the goal of the encounter is not to kill it, but to feed it a nice homecooked meal. (Think Sam Gamgee when you think of one of my PCs.) I have another PC that is dying to use his grappling hook so I was thinking of putting a giant lake in the middle of the underground cavern that the PCs have to cross. Would allow the PC some creativity to use his hook.

Somewhere in this cavern is a high-level artifact that the bad guys want, don't know it's here, and the good guys don't know what it does or what it is exactly. (They know the bad guys are after something but they know nothing about what the bgs are after.) So it's not really something they can use. But once they find it and have it, they will protect it. (It will probably get stolen at some point.) So it's reasonable that a valuable thing like this will be well-protected but I'm not sure it's not too much for a party of 5th level adventurers. (There'd be no combat encounters in the cavern. Maybe some traps/puzzles to get by but no outright enemies.)

It's totally ok to add natural caverns connected to a worked stone dungeon. There is nothing wrong with adding a passage that leads down into the dark too.

Regarding the artifact, it can be something that looks simple, but under XX conditions or event, it becomes something special.

As long as you got a plausible reason why something/someone is there... Then you can add as many layers to your environment as you like. It is ok to have environments that are heavy with exploration, puzzles and traps to break up the combat and rp sections. I think it is good to think in contrasts, such as they do with movie scenes. If one is dark then the next area/scene should be the complete opposite. That way you also get a rollercoaster of emotions and immersion going.

The question is more... What does your group like content wise. If they prefer combat then throwing a lot of puzzles at them is not a good move.

They're my nephews so they are used to video games that are just stab-stab-stab. That's why there's a lot of combat in the first level, and one big battle in the second. But I wanted there to be something different so that their dad (who also plays in the group) gets something outside combat and they see that there's more to the game than just stab-stab-stab. So while half the party is just murder hobos, the other half (me and my brother) would like to have them see that there's more to the game than just combat. I still lean combat-heavy and gloss over RP stuff cause they're kids, but I would like to vary it a little more. It's feasible they would get a long rest before exploring what's beneath the trap door in the labyrinth's central room. (They'll probably take a long rest after the first level of the dungeon. Otherwise it could be a TPK in the labyrinth.) And I know what the artifact is. It's a plot point so that doesn't need to be decided. But the one nephew is dying to use his grappling hook. The other spends his downtime in every city looking for more cooking supplies to purchase. So to present something that is a way to actually use those is appealing to me.