I Played D&D With AI Chatbots, And Here’s How It Went

With Microsoft Copilot’s free version now upgraded to GPT-4 Turbo, I wanted to experiment with it. I wanted to see if its ability to parse information, figure out my intentions, and remember my past actions had been improved with the upgrade. But most importantly, I wanted to see if it could let me play Dungeons and Dragons.

Having an AI chatbot play D&D is an interesting test because it checks to see if the AI can balance the logistics of the game system with the creativity needed to build a world. As such, I asked Microsoft Copilot to run a game of D&D for me and pitched it against its AI rivals to see which one was the best Dungeon Master.

Microsoft Copilot: a very impressive Dungeon Master

Not quite the real thing, but a close facsimile

To get started, I gave Microsoft Copilot a try. I found that if you didn’t set the conversation style to “more precise,” the chatbot had a tendency to throw the ruleset out the window and make things up as it went along. With the “more precise” setting turned on, Copilot played by the rules very well. It generated a character for me, assigned it proper stats, then set me down in Waterdeep and gave me a mission to retrieve the Star of Corellon after it was stolen by a thief called “The Shadow.”

There were a few continuity issues; for example, I was told the Star of Corellon would be sold at an auction at Baldur’s Gate in three days, but the journey took five days without any holdups. Despite that, the AI told me that I still had time before the auction, so perhaps keeping track of time wasn’t its strong suit.

Regardless, it handled the roleplay very well. I arrived at the auction and told the auctioneer that the Star was a stolen item. I convinced him that this was essentially selling contraband, which allowed me to negotiate the return of the relic. I also made an attempt to take down The Shadow, and while it didn’t let me roll to see if I hit with my arrow aimed at his leg, it still roleplayed the situation well.

Things began to unravel the further I went. When I retrieved the artifact and asked Copilot what there was to do in Waterdeep, it suddenly shifted into AI assistant mode, as if I was asking it about a holiday recommendation. It then got confused over my wanting to join the Emerald Enclave with me already being a member, and after that, the roleplay began unraveling. Still, I got an adventure in before it fell apart, so I was pleased with the end result.

If you’d like to see my adventure, check out the full log with Microsoft Copilot.

Google Gemini: more a storyteller than a roleplayer

An unconventional Dungeon Master

Google Gemini, on the other hand, didn’t really care so much about the D&D system past the start. It began smoothly, with the AI generating a character for me based on the rules of D&D. However, the moment the adventure got started, all of the rules and styles of leading a roleplaying game went out of the window. Instead, Google Gemini acts more like a story generator than a roleplaying leader.

For example, Google Gemini didn’t throw threats at me. Instead, it listed off potential threats my character could face and asked me to pick some. It felt less like the AI was guiding me through a story, and was more working with me to generate a story that we could both work with. It’s definitely not how D&D usually goes, but the end result allowed us to collaborate on a story.

Google Gemini was also keen to take the reigns and do all the roleplaying for me, and sometimes I only had to type a few words about what I wanted to do, leading the AI to generate entire paragraphs about what happened.

After a while, the AI began to unravel, much like Microsoft Copilot. After one fight in a canyon, the AI fixated on canyons for the rest of the game, with no action happening in any other biome. And it forgot that our mission was to defeat the orc warlord, and the stolen children we ventured out to save seemed to evaporate when we arrived at the orc camp. Despite this, I still had fun exploring the story, even if it wasn’t conventional roleplaying.

You can see my adventure in my Google Gemini log.

ChatGPT 3.5: the weakest of the bunch

Lagging behind the rest

Unfortunately, it seems that ChatGPT 3.5 didn’t manage to hold a candle to its competition. While it did offer roleplay in a more traditional sense, the adventure ended a lot quicker than the other stories. It also had some very weird moments, such as the person who gave me the quest suddenly appearing beside me to help me prepare for a siege on a pirate stronghold. And ChatGPT treated me killing the pirate captain as us vanquishing every pirate in the stronghold. Plus, while other systems never really used the stats they gave me, ChatGPT 3.5 straight up didn’t give me any.

You can see my short-lived adventure in my ChatGPT log.

Different styles for different folks

While I was originally going to compare each chatbot to see which one reigned supreme, I discovered that each one had its own nuances. Copilot was more likely to run a game like an actual roleplay, while Gemini was more of a collaborative effort between us both. And while ChatGPT was short, that may be a boon for someone who doesn’t want to slog through paragraphs of text. Either way, this was a fun experiment, and I’m keen to see how each AI will develop in the coming months.