In my campaign, we jokingly have a DnD Walmart(tm), but the issue is, we have magic items that I have NO idea how to price. Like, how much should a common magic item or a rare magic item should cost??

Here's a pretty decent guide.

I found the prices for healing potions to be a bit low, but the rest is a good starting point. Also, consider a variation on James Haeck's house rule for haggling. Price an item a bit high and allow the player to attempt to haggle. Make the player describe how the character goes about haggling. Then it's a skill check based on the character's persuasion, deception, or intimidation contested by the seller's insight. If the character wins, whatever they rolled is the % they convince the seller to lower the price by.

I'd say it depends on whether you want it to be affordable or not. How much gold do you plan on giving the players?? How hard do you want the magic item tier to be?? How affordable is this for the world at large?? Answer those questions and you should be fine!!