You could take inspiration from the mists of Ravenloft. They are essentially a plot device to keep people locked in the domains of dread under threat of something inscrutable yet terrible.
See the following from Van Richten's Guide to Ravenloft.
For example the mists of borovia have a life draining effect but also warp space so that you never get far from the edge. There are also monsters like vampiric mist which can infest your mists or things like cloudkill which can replicate toxic gas. There are lots of options
You don't need a specific thing that already exists. There's an island shrouded in/surrounded by. mist. It's not possible to reach the island.
You just need to decide a few things:
What is causing the mists?
Do they look unusual?
What happens when people sail into the mists? Do they pass through to the far side? Get turned around? Sent to another plane/part of the world? (Steadily damaging people may not be enough to stop determined PCs.)
How does one get through the mists? It needs to be something the PCs don't have access to. (You can decide later, using a method that hasn't been tried/wasn't possible, but better to know off the bat.)
Do the mists also block teleportation? High-altitude flight? Undersea travel? Tunneling through the earth?
If you’re the DM then such a fog exists because you say it does. You don’t even need to have any explanation for it unless you plan on it being something the players can figure out. Why it works or what causes it or anything like that is only as import’s you make it. The only thing that you really need to know about it is how it works so you can be consistent with it.
This is exactly the reason that a good quality Dungeon Masters' Guide was necessary before anything else for players in D&D.
A DM is responsible for building out the world. They decide how that world works. For example gravity in Faerun works distinctly differently to how it works in the Spelljammer Setting. Likewise in other planes. When you build the world, something works the way you say it works - no further explaination needed.
All of that said, there is a chance here you've got a problem similar to Vecna Eve of Ruin has - 5e player characters are so powerful that their spells and abilities can dismantle things the DM hides stuff behind real easy. In Eve of Ruin, the writers literally had to invent a new magic item that negated a character's spells and abilities that normally would have uncovered an NPC in disguise. So, simply inventing a fog that is inpenetrible places you in the same position as the writers behind the system.
Generally speaking though, this happens when a part of the world building has been done poorly. In WotC's case the weakness was, in my estimation, made in original design but discovered only during QA or test playing. So as a result they had to invent something last minute to fix that problem. In the OP's case my guess is that you don't want the player characters visiting the island 'early'. My question to this is: why not?
Maybe you're seeking to protect the party from an area that is stocked to the rafters with high level enemies. In which case, deterring the party is as simple as leaving clues to just how powerful the enemies are. If you provide a few warnings to the player characters that the island is too dangerous, but they decide to visit anyway that's a valid choice. If their characters die - oh well that's on them and a reasonable result of their choices.
So, I guess the best question to ask is: why do you want the island to be impenitrable to your player characters?
I am looking for fogs in dnd that could keep a evil island protected for now. Sorry I am bad at explaining things ;c
You could take inspiration from the mists of Ravenloft. They are essentially a plot device to keep people locked in the domains of dread under threat of something inscrutable yet terrible.
See the following from Van Richten's Guide to Ravenloft.
https://www.dndbeyond.com/sources/vrgtr/domains-of-ravenloft#TheMists
It can be
For example the mists of borovia have a life draining effect but also warp space so that you never get far from the edge. There are also monsters like vampiric mist which can infest your mists or things like cloudkill which can replicate toxic gas. There are lots of options
You don't need a specific thing that already exists. There's an island shrouded in/surrounded by. mist. It's not possible to reach the island.
You just need to decide a few things:
If you’re the DM then such a fog exists because you say it does. You don’t even need to have any explanation for it unless you plan on it being something the players can figure out. Why it works or what causes it or anything like that is only as import’s you make it. The only thing that you really need to know about it is how it works so you can be consistent with it.
Homebrew magic fog usually works, make it so it seems like they are getting closer but they are actually not.
This is exactly the reason that a good quality Dungeon Masters' Guide was necessary before anything else for players in D&D.
A DM is responsible for building out the world. They decide how that world works. For example gravity in Faerun works distinctly differently to how it works in the Spelljammer Setting. Likewise in other planes. When you build the world, something works the way you say it works - no further explaination needed.
All of that said, there is a chance here you've got a problem similar to Vecna Eve of Ruin has - 5e player characters are so powerful that their spells and abilities can dismantle things the DM hides stuff behind real easy. In Eve of Ruin, the writers literally had to invent a new magic item that negated a character's spells and abilities that normally would have uncovered an NPC in disguise. So, simply inventing a fog that is inpenetrible places you in the same position as the writers behind the system.
Generally speaking though, this happens when a part of the world building has been done poorly. In WotC's case the weakness was, in my estimation, made in original design but discovered only during QA or test playing. So as a result they had to invent something last minute to fix that problem. In the OP's case my guess is that you don't want the player characters visiting the island 'early'. My question to this is: why not?
Maybe you're seeking to protect the party from an area that is stocked to the rafters with high level enemies. In which case, deterring the party is as simple as leaving clues to just how powerful the enemies are. If you provide a few warnings to the player characters that the island is too dangerous, but they decide to visit anyway that's a valid choice. If their characters die - oh well that's on them and a reasonable result of their choices.
So, I guess the best question to ask is: why do you want the island to be impenitrable to your player characters?
DM session planning template - My version of maps for 'Lost Mine of Phandelver' - Send your party to The Circus - Other DM Resources - Maps, Tokens, Quests - 'Better' Player Character Injury Tables?
Actor, Writer, Director & Teacher by day - GM/DM in my off hours.