So maybe this has been discussed before and if so I apologize, but I am about to starting an Artificer at Level 4 and choosing the alchemist specialty. What can my character create with alchemy other than healing potions? Not that those aren't great, but can he create magic items and the like? And is this an official list with the ingredients required and information like that? 

Thanks in advance.

Xanathar's Guide has a section that elaborates mote on what the different tools can be used for. healing potions are actually made with the herbalism kit (which alchemists also get proficiency in).

Between the two you can:

1. Craft any potion as a downtime activity (quicker and cheaper than normal I might add).
2. Make alchemist's fire (flask), acid (vial), antitoxin, oil (flask), perfume (vial), or soap as part of a longe rest (with half GP worth of material that you can buy ahead of time).
3. Identify plants and potions.
4. Find plants
5. Identify poisons
6. Create a puff of smoke
7. Identify a substance
8. Start a fire (chemically)
9. Neutralize acid