In one campaign I'm planning to run eventually, I had an idea to make my main villain, a tyrant from a long-lost empire named Fomalhaut, completely mute in almost all of their interactions with the party or their subordinates. This silence is not due to reasons one might expect, though: Fomalhaut instead simply chooses not to speak to creatures 'not worth their time or effort', and seeing how arrogant and powerful they are, that applies to just about everyone. I wanted to run them as a rules-by-strength, clearly powerful villain that makes the party hesitate... but I can't think of many ways they can do that without resorting to cliches or cartoonishly violent actions.

How can I make them fill the role of BBEG without making them a two-dimensional generic bad guy?

Give him a family. Show his humanity. Insetad of him being a singular entity give him the depth of relationships that are worthy of his time and effort. Especially with people who aren't 100% evil themselves.

Yeah  by and large, to escape the two dimensions of "Evil" and "Cruelty" you have to have something that humanizes them, gives them an everyman type of flair (even if that everyman is a noble power).

It isn't going to be that hard, though -- he may not deign to speak to those below him, but he will have operated in a world where those below him do not know what he wants them to do unless he speaks -- so he is likely to have a few people he uses and possibly cares about what happens to them (Even if for no other reason than the inconvenience it causes him). These people will be the sort that know him well and are dedicated to him -- they can read his mood and thoughts with a glance, no words needed, and they almost always start their sentences with some variant of "he says, He wills, He finds".

Another possible option is that he is someone to whom rages are a problem. Minor setbacks, major setbacks, little annoyances -- all of them cause him to fly into a rage where he smashes things. In this case, those who help him are going to seek to stop from doing that, knowing that he can't help himself and that he breaks things of import and value to him. In their eyes, he will have been hurt by that.

The thing is, not all evil is uncaring, not all evil operates alone, and if you are doing complex plots and need deniability, you have to operate through others.

Make him write orders in Haikus, care about his underlings children, tending flowerarrangements, or bonsai trees, ok now that i write it all that samurai stuff is a good way to let him use few to no words. Let a simple "No." uttered by him impact as a tsunami, causing astonishment or terror in his people. All depending on how much contact the pcs get with him.

If they rule by strength, there must be signs of that strength somewhere, whether it be in stories or rumors or visible fallout of their actions. Have the party find the fortress of the BBEG's previous rival cleaved in half, something like that. A powerful tyrant is going to leave scars on the world, so show off the scars.

Two-dimensional isn't necessarily a bad thing. Unless the party interacts with him a lot, they're not going to see much hidden depth. (And the non-speaking thing makes that pretty unlikely.)

The thing you probably want to avoid is cliche. Cartoonish violence is bad. Controlled violence is good. The threat exists at any interaction with him, but it's rarely really foregrounded -- the PCs are not worth the time and effort. 

One possible approach would be that his philosophy is that you make the boundaries clear and the consequences clear, and any violation is dealt with with more-than-sufficient force. And then you make sure that the resulting fights are never, ever, pushovers. If the players know that crossing him is always a serious problem, they're going to respect him.

After all, evil doesn't have to be stupid, or wantonly violent. (That's what PCs are for. :) Maybe he publicly executes minions who are arbitrary, or who torture people.

There's a lot to work with with the non-speaking thing. He's going to need to communicate with his lessers to make things happen, so there have to be people who speak for him. Maybe he communicates with his subordinates with sign language in public.

There's probably layers of communication, with minions telling lesser minions, all the way down to the guard captain who says to his guards, with the PCs right there, "Tell them to move along, or they'll regret it." And the guards don't say anything like "You heard him!" -- they tell the PCs to move along.

And then, at the end, there's the payoff of when he finally, grudgingly, speaks to the PCs, just to tell them that they're going to die now.

On Villains: Villains are so interesting to play around with and make memorable. Several good suggestions up there about anchoring the villain into the world through family, community, and goals. Because goals are what makes the villain a Villain. Also, villains are at their best when they involve others into their scheme. Lone wolves might LOOK cool, but they have a hard time affecting enough of the world to ascend from (powerful) thug to proper villain status. Give them a network of well-meaning supporters, mind-thralls, or faceless blackguards.

To make them stand out, consider having them do "The right thing for the wrong reasons", or "The wrong thing for the right reasons". Also, is their goal to prevent something from happening, or to cause something to happen?

Maybe they want to prevent the spread of a deadly plague that turns people into undead, and orders the burning of entire villages to ensure this? Or perhaps they seek to break the curse of petrification on the Underkingdom, but only so that they can get their hands on the secrets of the Soul Weaver, lost since aeons among the ruins below?

Not Speaking: Make sure their actions speak for them. Really dig into the "Show, Don't Tell" motto. And be insidious about it, too! It's easy to show how a villain is strong by having them publicly fight and beat another strong NPC or monster. BUT! What if, instead, the characters see the effects of the villain's plan for the world. Maybe entire villages are now running with a new system of government, with new laws and magisters put in place by the villain's organisation? Harsh punishments for those who step out of line, but enough good things done to the community to keep them in that line.

"Say what you will about Lord Tyrant, but he makes the ship and wagons run on time. Sure, taxes are high, but since his Blackguards burned down Old Basilisk Forest, we have a huge new field for our farms that produce barley and cucumbers like you wouldn't believe!"

And then dial it up from there. The hobgoblin tribe threat you had the characters investigate in the beginning of the campaign? They're gone now. Incidentally, Lord Tyrant's borders now extend into the Shatterspleen Mountains where the hobgoblins used to live. What a coincidence! Threats that the characters have outgrown, but that are still present in the world? Use those as punching bags for the villain, but at scale enough that the players realise that there is something big going on here.

Have fun, and let us know how it went! :D

The only person the party sees him talk to is his dog. Guy loves loves loves his dog. 

And/or, give him telepathy. He “speaks” to people when necessary that way because opening his mouth is such a chore and people aren’t worth the effort.

And/or this one time, when he was 8, a bug flew into his open mouth and it really freaked him out. Most people are not worth the risk of that happening again.

Quote from Xalthu >>

The only person the party sees him talk to is his dog. Guy loves loves loves his dog.

 And Players better don't hurt that dog or he goes full John Wick on them?

Have the rumor BBEG executed some Underling or Noble like a rabid dog, because he was cruel to his people just for fun.

Have one or more NPCs the party knows and trusts attest to the villain’s power and ruthlessness. Since the PCs, and therefore the players trust the NPC(s) and respect their opinions, it will add weight to that BBE’s reputation.

When the PCs do see the BBE, have him inflict violence on someone, but not wanton violence, and don’t have him fly into a rage, if you do he’ll just come across as petty and look like he has temper tantrums. (Like Kyle Ren, he just seems like a big baby.) That’ll just goad the players into wanting to smack him down. Instead, have him commit cold, ruthless, calculated violence. (Like when Vader used the force to Choke Admiral Motti, he seemed like a stone cold, bad ass BBE.)

One of my favorite things is to have the villain sitting in state, judging someone for some minor crime.

the room is vast, and the players learn that it is early in the day as they look around at the brown, strangely varnished walls with holes in them, decorated with exquisite edging of great cost, two men with a chest, holding hammers standing near one of the wooden walls.

the BBEG listens to the crimes read off. The petty criminal sweats and shivers, terrified. At the end of the recitation, the BBEG gestures at the wall, bored.

his seneschal frowns, and turns to the witnesses. “He has found guilt, and declared a standard punishment. Nail him to the wall!”

whereupon the guards bodily lift the now screaming criminal, haul over to the two guys with hammers, and he is summarily nailed to the wall.

the brown varnish is old blood.

Well, here's one short thing to go into. 

If they simply speak to creatures that are worthy of even being spoken to, most people in Fomalhaut probably would have caught onto that, or at least gotten some cues. 

If they speak, even the shortest sentence should have an extremely large impact, and the general reaction should be somewhat terrified, either of who they are talking to ( again, if they're talking to someone, it's a huge deal ), what they're going to do, or a mix of both. 

As for cartoonishly violent actions, there's really no such thing if you have a good reason. A good backstory and well-built character development should give depth and reason to their actions, and cartoonish violence can suddenly transform into having a lasting impact on the campaign. 

Well, I probably didn't explain anything well, but whatever. Good luck!