My 5 lvl Bard casted a Phantasmal Force to a large Green Dragon (GD), it succeeded.

I created an illusory Red Dragon just in front of him. I also added an illusory wall just behind the illusory Red Dragon for vision purposes (supposedly). I filled the 10-foot cube with illusory treasures and gold, intending to motivate the GD fight with the illusory dragon and get his loot.

Is it obligatory for the GD to spend his action on my illusion? Or he can simply bypass the illusion and fight any member of the party?
Furthermore, what happens if I maintain the spell while the rest of the party attack him?

In total, I casted 3xPhantasmal Force and the GD was spending his actions to investigate the illusion, while my party was fighting him regularly. In the end the GD tried to flee and he flied. Unfortunately for him I casted Hypnotic Pattern and incapacitated him. He fell down and died by the fall damage.

After all, it was a pretty satisfying play for me but I feel the spell is a bit overpowered.

Did we play it correct?

Phantasmal Force is a powerful spell if the target fails the save (I'm not sure how a young green dragon failed so many saves, DM must have been rolling bad).

The GD could have ignored the illusion, but I don't think it would if a red dragon appeared in its lair with some treasure.

Was your DM remembering to use lair actions? Those can really change the feel of a monster.

Well...the GD may fail its save and believe the red dragon is there, but if the red dragon isn't attacking it, and other creatures are attacking it, there's no reason for the GD to completely ignore the creatures that are attacking and damaging it. Particularly with a breath weapon, which could take in some of the party and the illusion red dragon. From the info here, the GD wasn't played very well by the DM.

And, looking back at the spell...each casting gives you a 10' cube, and requires concentration. So you would be able to create the illusion of a red dragon...that was no taller than 10 feet, and no longer than 10 feet. In other words, a baby red dragon. And because it requires concentration, if you cast the spell again for another illusion, your dragon goes away, because only 1 concentration spell at a time. So you were also being allowed to abuse the spell. 

Phant Force is a good spell, but it's not as good as this. 

The original question seems unanswered. To answer it I say the GD would treat the party's attacks as blows by the red dragon and want to deal with it first before turning attention to the party.

Quote from IDNeon357 >>

The original question seems unanswered. To answer it I say the GD would treat the party's attacks as blows by the red dragon and want to deal with it first before turning attention to the party.

That's one possibility. The spell does indicate that with a failed save the GD would rationalize what is happening and believe the illusionary dragon is harming it. But that doesn't mean it still can't see what's going on outside of the spell's 10 foot cube area. The GD would see a (very small) red dragon in the area of the spell. And that could mean that someone standing in that cube could be absorbed into the red dragon illusion. But unless you can pack your whole party into a 10 foot cube, it's also going to see the party members who aren't in that area. The spell doesn't indicate that the GD would treat any damage it suffers at all as coming from the illusion. If the illusion is in front and you attack it from behind, that's way too powerful of a spell if it completely makes the GD unaware of what's actually behind it. 

I'd say that anyone damaging the GD from within the spell area would be rationalized by the GD as the red dragon attacking it. But it is still going to be able to see the ranger standing off to the side peppering it with arrows. And there's nothing that says it has to ignore that other opponent. And if the ranger is doing more damage to it than the red dragon is (which is 1d6/round without someone else's attacks being 'absorbed' into the spell effects), I'd certain have the GD start taking some attacks against that ranger if I was the DM.

In fact...the spell specifically states that the phantasm red dragon can only damage the GD if the GD is either within the spell's area (10 foot cube), or within 5 feet of it. That's going to be severely limiting. So here's a very likely scenario:

• Party of 5 players find GD. Wizard casts Phantasmal Force. GD fails save.
• GD sees a 10 foot tall red dragon. 2 of the party members can fit inside a 10' cube and still fight/cast spells/shoot arrows/etc. The other three if they want to attack, must be outside the spell's area. I rule that the 2 inside the area are hidden by the red dragon illusion. (You could argue maybe 3 inside that area...but 10' is really not all that much space, particularly if you're firing arrows, casting spells, swinging swords, etc.)
• So the GD sees a red dragon, and three other party members.
• If the GD uses it's breath weapon, or spells, and does not close to within 5'  of the red dragon illusion, the phantasmal force spell cannot harm the GD.
• So if that happens, the GD sees a red dragon it believes is there, but for some reason is not harming it (the fire of the red dragon does not extend out of that 10' cube, remember), and the GD also still sees the other three party members, who are attacking it, and hurting it.
• If the two hiding inside the 10' cube damage the GD, I'd rule that the GD would rationalize that as the red dragon attacking it, that's cool. But it still sees the other three party members.
• There's no reason for the GD not to treat this battle as an otherwise normal battle between a very small red dragon and three other beings. 
• Thus, particularly if the other three are doing serious damage, the GD should be attacking them as well. 

Even if the GD closes to within 5' of the red dragon illusion, the GD still gets to be smart. If the three people outside the illusion are doing more damage than the illusion--1d6/round--or the illusion + 2 hidden party members, the GD should still be thinking about attacking them.

And if the caster tries to add a second phantasmal force of loads of treasure, or a wall, the red dragon disappears.

Heres an idea. Make the party have to hold back their attacks so the red dragons seems a greater threat. If the party makes a stronger attack than you would expect from the red dragon. Then the GD will go for the larger threat.

This requires players to be smart about how they attack, going for just enough to keep harming the creature, not enough to aggro