Let me explain. I am making a tiefling paladin, and he was abandoned by his father on one of the nine layers of hell. He rose through the ranks be being charismatic, and became a high ranking officer in the army. He was then sent to the material plane to fight, but his army was destroyed and he was stuk in the material plane.

So, what background is that, like criminal or urchin or soldier? 

I would go with Soldier, but you should choose whichever one makes the most sense to you.

Your story says soldier, i could also see far traveler https://www.dndbeyond.com/backgrounds/far-traveler

Criminal or urchin don't really make sense as their written.  Your character was raised actually in a sort of privilege and attained a high rank.  While military training _may_ lead to the sort of skills those two backgrounds have, since you spent you back story in military pursuits inside the ranks of Hells armies, I don't see how you'd develop criminal contacts or an inmate knowledge of your game's city (unless of course you're talking about aspects of Hell, that you're currently cut off from).  

Solider too, the military order that would acknowledge your rank feature is destroyed on your game's material plane, and depending on your game's power level, the utility of the rank feature between your game's main locations and Hell may be problematic.

I could see Far Traveller working the best, your exotic background granting you access and audiences.

Soldier is the most obvious given your description, but I'd consider the feature of potential backgrounds as well - the Soldier's Military Rank probably won't do a whole lot for your character on the material plane, as MidnightPlat noted(that doesn't have to stop you, but it should be pointed out).

If you flesh out the backstory a bit there are quite a few more that could be a good choice though - Acolyte might work, or Knight or Knight of the Order. Pirate actually seems like a great option if you ignore (or reskin!) the tool proficiencies. Archaeologist could fit, if you work a bit on your upbringing in Hell - it doesn't have to be just military stuff. If I was your DM I'd jump at the opportunity to work out a "faction" that could help you on the material plane with the Faction Agent background. I suggest thinking out of the box a little bit, expanding your backstory and seeing if there isn't something better than the obvious for you.

I think I agree with the general consensus here that the Soldier background sounds the most appropriate.  Or maybe the Noble, if you want to explore the more political side of your character's high military rank.

HOWEVER, coming of age as the leader of an army of devils is going to come with some SERIOUS psychological baggage.  While I think the trope of the "emo, messed-up" character tends to be a bit overdone, in this case, I think it's HIGHLY warranted.  If it were me, I'd go full-blown PTSD with this toon, and let your DM leverage that for some interesting story hooks later.

Urchin would be fine considering this character sounds like they would be really high level at this point. Is this a level 20 one shot? 

There was a character in the old Buffyverse show Angel that was brought to Hell as an infant and came back to Earth as a adult teen/early twenty something IIRC.  Sort of in the emo mode, but became a pretty solid character on the show for inspirations. 

Also have to wonder what level the character is going to be if they're "high ranking" in the Infernal armies.  I mean in Descent into Avernus, bearded devils and equivalent are the foot soldiers of the stronger legions of Hell.  If you look at the order of battle there, a legion is the largest field unit (1000) Devils.  If you assume a bearded devil is the "baseline" unit, it's in a lance of ten probably commanded by something no weaker than a barbed or chain devil, that group is placed into a cohort I'm presuming led by a chain or bone devil, and those cohorts are put into a legion that's commanded bye a bone or a horned devil.  And then oversight beyond that you're starting with Furies Erinyes.  And this character is going to be a PC equivalent capable of leading at those power levels...

Other possibilities, the PC being a mortal isn't put in direct control over a military unit but is the "Hamilton" or aide de camp of a ranked fiend and is treated as the right or left hand of that leader. They've been through the infernal war colleges (part of a "forthcoming" project of mine including intelligence networks and displaced peoples and a frustrating system of pneumatic tubes, "forthcoming" in that it likely will show up in GMsGuild a year from now) ... maybe there's one on Avernus or that frozen level that some lore suggests is used as a military training/proving ground for forces going to Avernus.  Other option he was raised to be an unconventional operative.  So he aids military operations but he's more of a forward deployed scout/spy and functions as an advisor to leadership when back in friendly lines.  This could lead to the criminal spy background, the "underworld" contact probably being indebted to Hell for some reason.  Urchin is possible reskinned if you take the city knowledge as basically a lifetime learning the city through magical scrying, was basically bred over their lifetime to be a forward operator for Hell's offensive on the city until the planar crossing got botched.

So again, unless there's an easy passage between he planes, mechanically, soldier will be a sort of moot background because the rank they pull belongs to an army that went the way of the Bay of Pigs invasion ... on the other hand, if there is easy passage and say some militaries have infernal pacts where some of their officers attend infernal military academies, maybe the character could receive a reciprocal rank by one of those armies as courtesy.  Criminal spy and urchin speak to someone who's somewhere in between the military and intelligence apparatus of Hell.  I think the easiest one if he's basically the survivor of a shipwrecked invasion would be the Far Traveller. 

Thanks for all the suggestions, and think I will find a way to work the rank into the game, maybe by making the symbol on the uniform known in certain areas and cities. I will work something out. Thanks for your help.