So I know 5th ed doesn't just wholesale give monsters classes like back in 3.0/3.5 days, but I want to know how to do this for enemies.  What if I have an Orc clan that have all developed psychic powers, and now I want to stat up an orc Psi Warrior or Abbaration Bloodline Sorcerer?  What if there is a Rogue Mindflayer that has started training with a sword and has aspects of a dark, evil paladin?  What if there needs to be some more powerful Githzerai monks that represent some of the most focused and disciplined minds among the Gith?  Is there a way of doing this or do I just kinda have to guess and homebrew it myself with little guidance?

There is no way to actually add classes/subclasses to a monster. It would have to be done manually with the guidance given in the DMG’s DM’s Workshop section:

https://www.dndbeyond.com/sources/dmg/dungeon-masters-workshop#CreatingaMonster

I've been doing this a fair bit recently, and I think one thing that DMG doesn't really convey is that you shouldn't necessarily just add every class features as-is because it will make running a monster more complicated than it needs to be. Personally I try to simplify monsters as much as possible, and you can see this on a lot of official "class" monsters, especially the more recent ones.

For example, I created two very different monsters that have levels in Barbarian, and I wanted to include Reckless Attacks on both. But since one was a zealot who would never not attack recklessly I just made it a simple trait ("All attack rolls made for or against the zealot have advantage"). Whereas the other character (a mercenary commander) got the feature basically a player Barbarian gets it (so she can choose whether to go reckless or not).

If your monster is going to be a spellcaster, I find it helpful to think about what features and spells they need, and what can you bake into their profile? For example, I less recently created a boss who makes a point of protecting himself psychically with mind blank, but instead of adding the spell, I just added charmed and psychic damage immunity, then added a note in the description to the effect of "Mind Blank: <redacted> is affected by the mind blank spell rendering him immune to divination spells and having his feelings/thoughts sensed, in addition to the immunity to psychic damage and being charmed (included in his stat block)", this avoids the bookkeeping of casting the spell in advance, but gives you a note in case you need to handle dispel magic or any divination etc. This can apply to any persistent effect; for example, if a monster's AC will come from mage armor, you could just make it part of their AC and ignore the spell for simplicity, or add it as "Armor Class 12 (15 with mage armor)" just in case the players try to dispel.

Another common thing with spells now is including the important ones as actions, and instead of including multiple cantrips or spells you may consider just having more generic combined versions with less resisted damage types. For example, a lot of newer wizards profiles have an "arcane blast" that's basically eldritch blast that can be used in melee range as well, and serves as their main free attack. They also have an "arcane ward" that's kind of like an abjurer's ward but included in place of giving them shield so they have damage reduction, but not in a way that will annoy the players endlessly (as shield is way more powerful on a lone boss because they're never going to waste it).

Sorry if any of that seems more confusing rather than less (not my intention) but mercilessly grinding class features down into only the pieces I need/want has helped me to make monsters that I've found a lot easier to run; there's no guide to how to simplify features, I'd definitely keep in mind what you want from a monster with every feature you look at as you'll find some you simply don't need, or which can be added in a much simpler way.

So I guess the answer to your question is "homebrew with little guidance", though hopefully some of the above helped for ideas to simplify, and if you do come up with a block you can try copy/pasting it into a post here to get feedback and suggestions for simplifications.

Make two features that communicate your concept. You only need unique powers to come up once or twice in the fight to underscore the way you have described them.

Dark Paladin? Necrotic Smite attack and an aura that conveys a penalty to saves.

Psi Warrior? Telekinetic Push bonus action and Telepathy.

Enlightened Monks? Advantage on all mental saves, unarmed strikes have reach and deal force damage.

Not only is this simpler to run, but it's not constrained by the limitations of PC classes and subclasses. Monsters should be able to do things PCs cannot or the world starts to feel very small and same-y. Yeah, it's a little more work, but if you're in a pinch you can just grab two features from an existing class/subclass.

I have used ideas from the first edition,
where classes are "Heroic" magical powers.
and race-wide magical power is "Fantastic".
I consider it necessary to be A hero, to have "lost" a certain excess of Fantastic power.
Often considered "runts" by their people.