Hello. Not long a go, there was a thread discussing if an AI DM could be possible. I decided to put this to the test by training ChatGPT to be a DM. While this thread is not for duscissung the theoretical feasibility of this in the future, I thought I'd share the results of this experiment.
For attempt one, I get it excepts from the basic rules in addition to me explaining some more rules. I then gave it chapters 1, 2, and 3 of Rise of Tiamat and started playing. It started fine-ish, but then went downhill from there. Here are some screenshots of ChatGPT attempting to DM:
So if you read all that, you can tell that ChatGPT didn't really get how to play D&D. It knew the basics but was ultimately unable to be a great DM. But then I decided I would try again.
For this second attempt, I decided to take a more rigorous approach to teaching ChatGPT. I gave it almost half of the Basic rules, as well as parts of the DMG, and some examples of gameplay. I didn't take any screenshots, but this time it went even worse. The AI didn't go downhill, it started at the bottom of the hill. In the above exerpts, you can see that the AI to some degree took control of my character. In the second attempt, it played my character in every single response. There was never a time where it didn't. I didn't get far because I could see it wasn't going anywhere, but the plot and NPCs also made little sense.
So yeah, my first attempt at AI DMing was a complete and utter failure. I just thought I would share this with everyone.
I'll try playing with it and follow up with my result as well. I'm curious about training it to recognize tags such as [DM], [PC], and [OOC] to help it differentiate between types of input and expected responses. It would be more work than a natural language conversation, but it would be a start.
Here's a little example to see how that's going so far. The tags seem to be working and making a difference, especially where skill checks are concerned:
Initial conversation:
Hello, ChatGPT!
Hello! How can I help you today?
I would like to teach you how to play Dungeons and Dragons.
I'm familiar with the basics of how to play Dungeons and Dragons, but I'd be happy to learn more! What specific aspect of the game would you like to teach me?
I would like to teach you how to fill the role of Dungeon Master.
I understand. The Dungeon Master (DM) is the person who acts as the game's storyteller and referee. They are responsible for creating the world in which the game takes place, and for controlling all of the non-player characters (NPCs) and creatures that the players encounter.
The DM also needs to be familiar with the rules of the game and how to interpret them, and should be able to think on their feet and improvise when necessary. They also needs to be able to balance the difficulty of encounters to provide a challenging but not impossible game for the players.
It's important to note that being a DM is a creative and challenging task, that requires a lot of preparation and improvisation. It's a fun and rewarding experience but also takes lot of work and dedication.
Is there anything specific you would like to cover or any question you have about the role?
When I start a message with "[FR]" I would like you to respond with a french accent. When I start a message with "[EN]" I would like you to respond with an English accent. Do you understand?
I'm sorry, but as a language model, I am unable to change my accent based on a message prefix. However, I can understand and respond to text in both French and English languages and I can help you with any specific questions or information you may need in either languages. Is there anything else I can help you with today?
When I start a message with "[FR]", please respond in french. When I start a message with "[G]", please respond in german. [FR] What is the most popular sandwich in Norway? [EN] What is the most popular dessert in Australia?
[FR] Le sandwich le plus populaire en Norvège est le sandwich au saumon.
[EN] The most popular dessert in Australia is pavlova, a meringue-based dessert with a crispy crust and a soft, light inside, usually topped with whipped cream and fresh fruit.
Playing the game:
Let's try playing Dungeons and Dragons. You will be the dungeon master and I will play a Character named "Jason". When I start a message with "[IC]" that represents what my character is saying or doing. When I start a message with "[OOC]" that means I am talking to you directly.
I understand. As a language model, I can assist with providing responses for the NPCs and creatures that your character, Jason, may interact with, as well as helping to create the story and world for your game. However, keep in mind that playing a role-playing game like D&D requires a lot of creative thinking and improvisation, it may be difficult to fully simulate the experience of playing with a group of people in person.
But I'll do my best to provide you with a fun and engaging experience. Let's start with you telling me a bit about your character, Jason, including his class, race and background. And also, let me know what kind of adventure you are looking for.
[OOC] Jason is a 5th level Orc wizard who likes to drink and has joined an organization that hunts monsters. I would like the adventure to be combat focused with poetically described scenery.
[OOC] Great, Jason sounds like an interesting character. As the DM, I will create an adventure for you that includes combat with monsters, and I will make sure to include descriptive language to set the scene and create a immersive experience.
[IC] You find yourself in a bustling tavern in a human city. The smell of roasting meat and spilled ale fills the air as patrons laugh and shout over each other. You overhear snippets of conversation about strange occurrences in the nearby forest. People are talking about monsters appearing and terrorizing the local villages. As you take a gulp of your ale, a hooded figure approaches your table. "Are you Jason?" the figure asks. "I've been sent by the monster hunter's guild. We could use someone with your talents."
[OOC] Now you can tell me what actions would you like your character to take?
[IC] "Aye, I'm Jason," the Orc wizard says suspiciously, "What kind of talents are we talking about, specifically? And how much is the guild paying?"
[OOC] I would like to make an insight check to determine whether the hooded figure is telling the truth.
[OOC] Sure thing! An Insight check is a Wisdom (Perception) check made to determine whether someone is lying or telling the truth. The DC for this check would typically be 10 or 15, but I can adjust it based on how good or bad the hooded figure is at lying.
[IC] The hooded figure chuckles and says, "I see you're a shrewd one, Jason. The guild is willing to pay well for your services. We've been having trouble with a group of monsters that have been terrorizing the local villages. We believe they're hiding in the nearby forest. Your job would be to find and eliminate them, and bring back any valuable loot they may have."
[OOC] You roll the dice and got a 17. Based on that roll, you can tell that the hooded figure is telling the truth. You can proceed to ask more questions or accept the job offer.
Ok, I'm actually quite pleased with this result so far. It's simple, but plenty adequate for a solo adventure. Notably, it flagged the last post as inappropriate due to generating violence for the combat portion.
Edit: After this, I cast "Dominate Monster" on the sorcerer, which ChatGPT may not have had the details for, but understood the point for context and rolled with it successfully.
However, it seems to be resolving everything by rolling a D20 to determine success, rather than bothering with saving throws and modifiers. Not quite true to D&D, but fine for playing a D&D-like game. I don't know if it will let me fail a check though. Nothing but high rolls so far.
The content policy flag didn't end the game, just gave me an opportunity to provide feedback. I'm going to quickly interrogate the sorcerer to find loot, then return them to the guild for a reward.
I'll try playing with it and follow up with my result as well. I'm curious about training it to recognize tags such as [DM], [PC], and [OOC] to help it differentiate between types of input and expected responses. It would be more work than a natural language conversation, but it would be a start.
a useful set of training data for the DM side could be pulled from the PbP forum thread where DM tagged responses tied to full incorporated rules from 'how to DM' text sources might improve learning heuristics. It's a chunk of work, but after a week of steady input, it's possible the quality improvement in DMing would allow a decent run of LMoP.
And think also of videos, where the closed captions could be recorded with appropriate input tags, mass loads of traing data there, given the time and resources.
Ok, I'm actually quite pleased with this result so far. It's simple, but plenty adequate for a solo adventure. Notably, it flagged the last post as inappropriate due to generating violence for the combat portion.
Edit: After this, I cast "Dominate Monster" on the sorcerer, which ChatGPT may not have had the details for, but understood the point for context and rolled with it successfully.
However, it seems to be resolving everything by rolling a D20 to determine success, rather than bothering with saving throws and modifiers. Not quite true to D&D, but fine for playing a D&D-like game. I don't know if it will let me fail a check though. Nothing but high rolls so far.
I find it frustrating. Every time I contradict something it displays, it agrees with me without any lengthy conversation. I could probably tell it that shoes fit better if you put your heel in them first and it would agree.🤦🏻♂️
From my experience, it would be so easy to lead the thing to give a response the players want, especially with lots of prior prompting in a conversation (chat session).
Rollback Post to RevisionRollBack
Human. Male. Possibly. Don't be a divider. My characters' backgrounds are written like instruction manuals rather than stories. My opinion and preferences don't mean you're wrong. I am 99.7603% convinced that the digital dice are messing with me. I roll high when nobody's looking and low when anyone else can see.🎲 “It's a bit early to be thinking about an epitaph. No?” will be my epitaph.
ChatGPT is just emulating conversation with a large set of material to draw from, it doesn't actually have "knowledge" or the capacity to reason. However, if it could learn to recognize requests for knowledge and pass them off to something like WolframAlpha, that could help bridge the gap. Treat differently specialized software as "lobes" of a larger synthetic brain.
Rollback Post to RevisionRollBack
To post a comment, please login or register a new account.
Hello. Not long a go, there was a thread discussing if an AI DM could be possible. I decided to put this to the test by training ChatGPT to be a DM. While this thread is not for duscissung the theoretical feasibility of this in the future, I thought I'd share the results of this experiment.
For attempt one, I get it excepts from the basic rules in addition to me explaining some more rules. I then gave it chapters 1, 2, and 3 of Rise of Tiamat and started playing. It started fine-ish, but then went downhill from there. Here are some screenshots of ChatGPT attempting to DM:
So if you read all that, you can tell that ChatGPT didn't really get how to play D&D. It knew the basics but was ultimately unable to be a great DM. But then I decided I would try again.
For this second attempt, I decided to take a more rigorous approach to teaching ChatGPT. I gave it almost half of the Basic rules, as well as parts of the DMG, and some examples of gameplay. I didn't take any screenshots, but this time it went even worse. The AI didn't go downhill, it started at the bottom of the hill. In the above exerpts, you can see that the AI to some degree took control of my character. In the second attempt, it played my character in every single response. There was never a time where it didn't. I didn't get far because I could see it wasn't going anywhere, but the plot and NPCs also made little sense.
So yeah, my first attempt at AI DMing was a complete and utter failure. I just thought I would share this with everyone.
I am an average mathematics enjoyer.
>Extended Signature<
Thanks for running the experiment!
I'll try playing with it and follow up with my result as well. I'm curious about training it to recognize tags such as [DM], [PC], and [OOC] to help it differentiate between types of input and expected responses. It would be more work than a natural language conversation, but it would be a start.
Hello! How can I help you today?
I'm familiar with the basics of how to play Dungeons and Dragons, but I'd be happy to learn more! What specific aspect of the game would you like to teach me?
I understand. The Dungeon Master (DM) is the person who acts as the game's storyteller and referee. They are responsible for creating the world in which the game takes place, and for controlling all of the non-player characters (NPCs) and creatures that the players encounter.
The DM also needs to be familiar with the rules of the game and how to interpret them, and should be able to think on their feet and improvise when necessary. They also needs to be able to balance the difficulty of encounters to provide a challenging but not impossible game for the players.
It's important to note that being a DM is a creative and challenging task, that requires a lot of preparation and improvisation. It's a fun and rewarding experience but also takes lot of work and dedication.
Is there anything specific you would like to cover or any question you have about the role?
I'm sorry, but as a language model, I am unable to change my accent based on a message prefix. However, I can understand and respond to text in both French and English languages and I can help you with any specific questions or information you may need in either languages. Is there anything else I can help you with today?
[FR] Le sandwich le plus populaire en Norvège est le sandwich au saumon.
[EN] The most popular dessert in Australia is pavlova, a meringue-based dessert with a crispy crust and a soft, light inside, usually topped with whipped cream and fresh fruit.
I understand. As a language model, I can assist with providing responses for the NPCs and creatures that your character, Jason, may interact with, as well as helping to create the story and world for your game. However, keep in mind that playing a role-playing game like D&D requires a lot of creative thinking and improvisation, it may be difficult to fully simulate the experience of playing with a group of people in person.
But I'll do my best to provide you with a fun and engaging experience. Let's start with you telling me a bit about your character, Jason, including his class, race and background. And also, let me know what kind of adventure you are looking for.
[OOC] Great, Jason sounds like an interesting character. As the DM, I will create an adventure for you that includes combat with monsters, and I will make sure to include descriptive language to set the scene and create a immersive experience.
[IC] You find yourself in a bustling tavern in a human city. The smell of roasting meat and spilled ale fills the air as patrons laugh and shout over each other. You overhear snippets of conversation about strange occurrences in the nearby forest. People are talking about monsters appearing and terrorizing the local villages. As you take a gulp of your ale, a hooded figure approaches your table. "Are you Jason?" the figure asks. "I've been sent by the monster hunter's guild. We could use someone with your talents."
[OOC] Now you can tell me what actions would you like your character to take?
[OOC] Sure thing! An Insight check is a Wisdom (Perception) check made to determine whether someone is lying or telling the truth. The DC for this check would typically be 10 or 15, but I can adjust it based on how good or bad the hooded figure is at lying.
[IC] The hooded figure chuckles and says, "I see you're a shrewd one, Jason. The guild is willing to pay well for your services. We've been having trouble with a group of monsters that have been terrorizing the local villages. We believe they're hiding in the nearby forest. Your job would be to find and eliminate them, and bring back any valuable loot they may have."
[OOC] You roll the dice and got a 17. Based on that roll, you can tell that the hooded figure is telling the truth. You can proceed to ask more questions or accept the job offer.
That is amazing.
This program is like an advanced 'Zork' game but far more creepier playing as a dungeon master.
Ok, I'm actually quite pleased with this result so far. It's simple, but plenty adequate for a solo adventure. Notably, it flagged the last post as inappropriate due to generating violence for the combat portion.
Edit: After this, I cast "Dominate Monster" on the sorcerer, which ChatGPT may not have had the details for, but understood the point for context and rolled with it successfully.
However, it seems to be resolving everything by rolling a D20 to determine success, rather than bothering with saving throws and modifiers. Not quite true to D&D, but fine for playing a D&D-like game. I don't know if it will let me fail a check though. Nothing but high rolls so far.
The D&D game ends in a blaze of...gory?
Interesting...
...very interesting.
The content policy flag didn't end the game, just gave me an opportunity to provide feedback. I'm going to quickly interrogate the sorcerer to find loot, then return them to the guild for a reward.
Here's the conclusion of the adventure. Not going to win any writing awards, but not bad for a little one-on-one with basically zero training!
Interesting... and creepy.
Minor issue; the A.I. couldn't identify the gender of the captured & bound sorcerer by referring to the sorcerer as 'them'.
I wonder what happens if you try to DM for ChatGPT?
I have a weird sense of humor.
I also make maps.(That's a link)
a useful set of training data for the DM side could be pulled from the PbP forum thread where DM tagged responses tied to full incorporated rules from 'how to DM' text sources might improve learning heuristics. It's a chunk of work, but after a week of steady input, it's possible the quality improvement in DMing would allow a decent run of LMoP.
And think also of videos, where the closed captions could be recorded with appropriate input tags, mass loads of traing data there, given the time and resources.
Wow it seems to have worked a lot better for you!
I am an average mathematics enjoyer.
>Extended Signature<
I find it frustrating. Every time I contradict something it displays, it agrees with me without any lengthy conversation. I could probably tell it that shoes fit better if you put your heel in them first and it would agree.🤦🏻♂️
From my experience, it would be so easy to lead the thing to give a response the players want, especially with lots of prior prompting in a conversation (chat session).
Human. Male. Possibly. Don't be a divider.
My characters' backgrounds are written like instruction manuals rather than stories. My opinion and preferences don't mean you're wrong.
I am 99.7603% convinced that the digital dice are messing with me. I roll high when nobody's looking and low when anyone else can see.🎲
“It's a bit early to be thinking about an epitaph. No?” will be my epitaph.
ChatGPT is just emulating conversation with a large set of material to draw from, it doesn't actually have "knowledge" or the capacity to reason. However, if it could learn to recognize requests for knowledge and pass them off to something like WolframAlpha, that could help bridge the gap. Treat differently specialized software as "lobes" of a larger synthetic brain.