I've found that it's hard to make a dungeon really work for level ones, as you get through maybe three encounters before you absolutely need a long rest, and that makes for a really short dungeon. I feel like level three is a good spot to start dungeon delving, especially since that's about when boss fights become viable as well, but I'd like to hear the thoughts of those more experienced then me, or even if you've found a way to make a level one or two dungeon feel satisfying without tpking 60% of the time.

Think you hit it on the head. The first 2 levels are for learning your characters. You can see by the exp requirements that they are intended to go by relatively quickly and 3 is when you really settle in. Of course, you can do a mini dungeon before that which may have only 3 encounters or so.

I find that some experienced groups prefer to just start at 3 for this reason. Other groups really like the vulnerability and desperation that those earliest levels provide.

A level 1 dungeon should have fail states that don't necessarily involve death, thus avoiding TPKs.

If a trap springs, or they fail to disable it, don't make damage the outcome.  Magical effects like "you now shed bright light at a radius of 10 feet and dim light at 30 feet" can really mess with future stealth attempts deeper in the dungeon, but doesn't necessarily mean the party is weakened in combat.

If the dungeon is the gullet of some humongous monster, its inhabitants are trying to expel you, not kill you, because maybe the host dungeon monster finds humanoid blood sickening.  If everyone drops to 0 hp, then the party finds itself in a puddle of slime and bile at the mouth of the dungeon.

Level 1 dungeons are also best if they are small, but depending on how you want to interpret "delving", can still be a feasible and satisfying experience.  A single room with multiple steps to unlock treasure can be a challenge for a low level group without straining their resources.

Also, if I build a low-level dungeon where a long rest would potentially enter the players' minds, I will typically insert some kind of "refreshment room" where those who drink of the mysterious, foaming liquid in the chalice in the center of the room gain all the benefits of a long rest, but will take 1 level of exhaustion in 1d4 hours, or a chamber with a glowing stone floating over a still pond that offers similar benefits.

It really depends on how you want the dungeon to be.  Not every room has to have a combat encounter;   puzzles, traps, and rooms to investigate/explore can fill out a dungeon without overwhelming your players.   The last dungeon I ran for my players (who were level 2) was 12-13 rooms (depending on how you count) with 3 normal combat encounters (undead kobold varieties) and a boss fight (an Allip the BBEG unleashed as a distraction before teleporting out).  The dungeon was themed as a rescue mission in the aftermath of an attack at an arcane university, so I was able to include a lot of skill checks (to find, free, and medically treat prisoners/captives), a puzzle (overriding a portal based security system), and several areas to investigate clues on the attackers.  I also introduced a captive professor of divine magics (using a priest statblock) they could free who would offer some healing in a tough spot (I let one of the players run him in combats)

I think as long as you offer some means of additional healing (through NPCs, scavenged potions, etc), you can make even a combat intense dungeon work at lower levels.

A party could get from level 1 to level 2 just walking from town to the dungeon, encountering wolves and giant rats and things like that... if you're that worried about it.

But I had my level 1s fight skeletons and kobolds and zombies and it was no problem.

You can run a "dungeon" setting at any level. However, the dungeon has to be scaled to the level of the party. You don't have parties in level 10 dungeons when they are level 5 unless you have given them sufficient warning they can decide to flee if needed. Similarly, you don't put level 1s into a level 3 dungeon but the setting itself can work at any level of character.

A level 1 dungeon might be an old abandoned structure/mine. Perhaps there is a lost child they need to find who went exploring. There might be some rats/spiders/bats/snake or similar natural hazards. Perhaps a hidden door or chest containing something long forgotten. There might be one or two situations requiring a fight, maybe a social situation or two depending on the nature of the dungeon, exploration and possibly a puzzle along the way to the conclusion. All of this is doable for level 1s. A larger dungeon would be more sparsely populated and might have rooms that could be closed off to take a long rest if needed.

One common issue with some dungeon designs is that they are too populated. Each room has encounters and sometimes these are so close together that any noise should trigger reactions from multiple rooms but which then would put the party in an untenable situation and likely TPK. So logical dungeon design scaled to level is an important element for any adventure. However, if you want to run a small dungeon adventure for level 1s there is no fundamental reason making that unfeasible.

This is what Goblins are for. Weak enough to not be a major threat to the party, but intelligent enough that, just in case the party all gets knocked out, that they'll just keep them as prisoners instead of just swarming and eating them. Also intelligent enough that, if the party really wants, they could avoid combat entirely and resolve conflict with negotiations or intimidation.

If you wanted to work a dungeon for low level characters, you could also design the dungeon such that leveling occurs before the dungeon is completed. This way the dungeon can start out harmless, but get more dangerous as the party delves deeper and have gained some XP.