It's basically a true pirate who made a deal to gain more powers.

It's a high Dex build with +3 Con and +2 Cha. 

Lizardfolk gets me the high AC without actual armor

Warlock - Pact of the Blade, Improved Pact Weapon and Devil's Sight invocations, limited spells including shadow blade, Fathomless for the tentacle, looking at it as a controllable ally for sneak attacks I wouldn't otherwise get. Also good thematically.

Swashbuckler - free disengage from enemies I've attacked, ability to draw attacks from allies.

ASIs/Feats

1. Skilled Expert - +1 Dex and proficiency and expertise in Survival

2. +2 Dex (20)

3. Metamagic adept - quicken and extend

4. Dual wielder - +1 ac circumstantially (18 or 19)

5. Resilient Con - +1 Con (16)

6. Tough

The idea is to not need to physically wear armor or carry physical weapons. I'm not sure that really gets me anything other than a thematically interesting character. The metamagic allows me to buff without losing a turn to do so.

Sounds like a solid build. I think you might have gone a little too heavy on Feats... if you roll for stats and get overall reliable stats my criticism is moot, but I think it's best not to plan too much for feats, especially since you'll have to be playing a very long running game to finally get to the Tough Feat. I find it's generally best to plan for one or two feats to really customize your character.

I'd also mention that, depending on how strict your DM is about what counts as an "enemy", the Fathomless Tentacle doesn't count as Enemy for the purpose of fulfilling the conditions for Sneak Attack. It's not actually a proper "creature" that you summon, but is treated as an effect that rests on the board. That said, for a Swashbuckler you've already got a big boost to Sneak Attack options, so you're not hurting... and if your DM doesn't allow your tentacle to qualify as an enemy for sneak attack, then it also doesn't count as a creature to count against your Rakish Audacity, so you could still use it as a barrier to keep others from getting too close and breaking your chance for Rakish Audacity sneak attacks.

I'm not sure the Fathomless tentacle counts as an ally for the purposes of Sneak Attack, but as a Swashbuckler you have an easier time qualifying for Sneak Attacks anyway. Have you considered Hexblade instead of Fathomless? As a lizardfolk you already have a swim speed and going Hexblade will give you the Shield spell and allow you to concentrate on Charisma over everything else.

I thought about going hexblade but part of this build that I like is not having to physically wear armor.  It's the while reason for going lizardfolk, meaning it has to be a Dex build.

Would it make more sense to go soul knife and pact of the chain warlock? Maybe Genie warlock, probably marid, or undead warlock.

Plenty of good Dex-based skills for a pirate. I wouldn't worry about that too much. 

I don't think there's really any need to plan out beyond that 3rd ASI already though. So much stuff can happen over those earlier levels, who knows if you'll care about those feats at that point. You have a solid concept, stick with that but decide on the details when the opportunity comes up.