Today, I received an invitation to complete a survey for D&D Beyond...a site I find useful for gaming, easy to use, and was invaluable during the pandemic since our group, which plays several times a month, never had to miss a chance to play at our favorite hobby. I appreciate the site well enough to buy a subscription. So I was happy to agree to help out with survey questions.
The first question requested my age. After filling in the box, the survey 'thanked me' for participating and ended.
Was it because I, truthfully, admitted to being 76 years old?
I've been playing D&D for over forty years, from 2nd edition on up, as both player and DM. I've spent....probably thousands...on rule books, modules, conventions and dice. I've also play-tested many other games and keep coming back to this one....why? Because I love it!
I'm truly disappointed that D&D Beyond doesn't think my thoughts on the game and opinions matter anymore.
On the other hand I got bored at about 25% progress after endless personality and targeted marketing and demographic questions. I thought I was going to be able to give feedback on the site itself... maybe after another 15 minutes of marketing questions.
Surveys usually have quotas for how many replies they need of given demographics and it's much more likely that the age demographic you fall into had reached its quota. It's not that your opinions don't matter because of your age, it's just that surveys have limits on how many responses they look for.
The survey was heavily skewed and all about the VTT. For one section you HAD TO pick which option was the “most compelling,” even if you found none of the options compelling at all, and it asked what websites we frequently used, but didn’t include crowd favorites such as DM’s Guild and Hero Forge among the drop-down choices. It was a very badly designed survey.
I received an invitation to a survey from D&D Beyond about a virtual table top that started off being annoying by claiming its contents are a secret, which is a completely unenforceable claim, then became more painful and time consuming until the questions continued repeating over and over in varying combinations and were un-skippable. At that point I decided the survey was outright harassment and closed it a little more than half way through. Whoever the social scientist is that designed the survey should be fired. Real people are not your Monte Carlo simulation tools.
This is the worst designed survey I have ever seen. That includes surveys that were so vague and ambiguous that the results were nearly meaningless. Harassing customers does not make them want to re-subscribe to D&D Beyond's services and almost convinced me to request cancellation and a prorated refund.
It's how surveys operate; you break your survey down into demographic groups and then aim to get an equal number of responses per grouping. This gives you a more representative breakdown across demographics rather than just eschewed towards the dominant demographic.
You can make presumptions of malice if you wish, but I can assure you as someone who is married to a research doctor who conducts a lot of surveys and studies (and based her thesis on it), it's a normal survey approach.
There are lots of interesting survey tricks that people aren't aware of. For example, have you ever wondered by the same question may come up two or three times, but the answer key seems shuffled around? This is a confidence assessment tool. If someone is engaging with the survey genuinely (ie answering honestly and thoughtfully) the answers on those repeat questions will line up more often than not. However, if the person is just clicking through without really thinking, those same questions will have different answers more often than not. This can indicate how much confidence can be had with the survey data.
There are lots of little tools that are used to make the data as useable as possible beyond just collecting as much information as possible.
I made it into the 55-64 group but found the survey to be.. iffy at best.
outside of general "what do you plan and how do you play games, rpgs" and "website's you use" (a lot of marketing or weird way to break down later answers), it was all VTT. Rewatch the video we've seen on it and tell us what you thought - and some horribly done rankings: picking compelling or unique across 3 options.. which were really around 8 just in different groupings with no option to say "nothing is compelling nor unique".
Personally I still find the VTT gimmicky. a lot of pretty miniature images and spell graphics but they keep talking about 'simple' and, to be honest, even simplified 4e (horrible version) seems more complex than their VTT look. I keep getting the feel the dumbing down of familiars and druids (really one set of stats per but different looks) is all about trying to squeeze it into their VTT. I doubt it will do anything like Roll20, much less FGU given what they've shown us. Heck, they can't even get their character sheets to do non-standard attacks correctly (no divine strike for paladin, no selection on weapons with multiple damage options).
I'm pretty well aware of how surveys operate. I mean gaming isn't all I've done for the last 40 years. And since I clicked on the survey invitation as soon as I saw an email had popped up in my multiple open windows, my conclusion is that the number of people in my particular 'demographic' was 0.
I would have loved to discuss ONE VTT TO RULE THEM ALL, considering how important and handy online rpg became during the pandemic. Right now, to play our campaigns, we are using a combo of three apps....D&D Beyond for character sheets, Roll 20 for game maps, and Discord, because Roll 20's audio is lousy. We've attempted to use Avrae for logging purposes, but about half of us can't seem to connect properly, in spite of the fact that our D&D Beyond accounts are linked to Discord.
At any rate, I appreciate all of your comments! Gamers are the best!
[...]I can assure you as someone who is married to a research doctor who conducts a lot of surveys and studies (and based her thesis on it), it's a normal survey approach.
There are lots of interesting survey tricks that people aren't aware of. For example, have you ever wondered by the same question may come up two or three times, but the answer key seems shuffled around? This is a confidence assessment tool. If someone is engaging with the survey genuinely (ie answering honestly and thoughtfully) the answers on those repeat questions will line up more often than not. However, if the person is just clicking through without really thinking, those same questions will have different answers more often than not. This can indicate how much confidence can be had with the survey data.
I hope your wife and others realise that using such "tricks" also poisons the well. If the survey asker doesn't compensate me for my time in some way (and DDB generally doesn't, not even like a freebie character sheet frame or something, and most surveys don't these days), then they're doing it on my generosity and on my charity. I'm willing to do that and generally put in a good faith effort, but when they're asking me the same question over and over...it just tells me that not only do they not value my time very much...but they're actively wasting it. On a good day, it'll put me off and I'll be less likely to engage with the content. I haven't done it yet, but catch me on a bad day and I'll start wasting their time in return.
The worst one I've had,.and I believe it was a DDB one, literally had entire sections repeated. Half the time I spent on that survey was repeating myself...sometimes several times. I still answered in good faith...but I've lost patience with it now.
Rollback Post to RevisionRollBack
If you're not willing or able to to discuss in good faith, then don't be surprised if I don't respond, there are better things in life for me to do than humour you. This signature is that response.
I had someone refer me to this entry. I was also dropped after posting my age.
The last 25 years I worked for a not-for-profit defense company that consisted of 80 scientists with “Doctor” in front of their names who did human research things like surveys.
About 1/3 of surveys were for national governments, state governments and individual companies across the planet. US clients include DOD, CIA, FBI, DEA, DOJ, DOT, Departments of Commerce, Interior, Veterans Affairs, and the list goes on. The surveys your kids and teachers get in High school have a good chance of being designed by that company (42 of the 50 states).
Survey update training by big name companies for collection methods and results and lessons learned from our surveys were weekly and monthly. Nothing enrages a client like angry survey respondents who would call their member of Congress over a survey we created for them.
I took an active part of survey creation, evaluation, distribution, response handling and data distribution using our own data, email and web servers until we went to using companies like Survey Monkey.
I mention this wall of text to point out I know surveys, from creation to collection to analysis of data.
It is a very poor survey that cuts off responses based on any factor on collection. The only valid way to do it is after data collection where it’s a simple toggle on an output but you still have the data for later analysis.
The fact they excluded based on age shows just how poorly the client understands the people who buy their products and how badly assumptions on the user population and data were built into the survey.
A VTT is not a video game and this survey may be an example of the “If you only have a hammer everything begins to look like a nail” problem when the background of those in charge comes from video games and not role-playing games. A constant refrain was “Check your perspective” when designing a survey.
Role playing gamers are different from video game players even if many users are in both populations.
Only 6 percent of video gamers are over 65 in 2022. This is an accurate hard number from places like Statista. In several sloppy online databases, who I will not name, this statistic is improperly labeled “Role Playing Gamers” Is it an accident the survey stops if you enter 65 or older?
The percentage of Role-Playing Gamers at 65 and older is actually 17% in 2022. This is roughly 1 out of every 6 players and almost 3 times that of video gamers. Considering the greater percentage of DM’s vs just players over 65 and the fact they purchase a much larger percentage of products for their groups shows Wizards should be checking their perspective.
A successful survey should understand its target audience and make sure they collect the correct data for the customer to make informed decisions on what they are trying to do. I think that was not accomplished by everything I have read and I bet the fault starts with incorrect consumer assumptions.
My second observation is that a VTT cannot compete with a video game and will only succeed with satisfied DM’s who let their groups use it and this survey dropped them when they entered an age past 65...
I stopped answering after it started asking me personality type questions and questions about my anxiety levels...i know we live in a time where everyone wants everything about them online, but goddamn, no thanks.
Yeah, I agree. Our DM has been posting the rule changes proposed for the next edition and asking us to respond.
All seem designed to nullify or decrease any species or class advantages that are part of the existing rule set. My first question is always: "You're not planning to implement this in our games, are you?" Fortunately, his answer has always been "No."
I try not to make the same kind of age-biased comments about the young that are generally lodged against older people. But in my mind's ear I can hear complaints that begin in a whining tone and amount to: "But that's not fair....."
IMO, the only question that should be asked is; "How does this make the game more fun?"
In this case, I believe that they don't think anyone who worked as a Compuserve admin, back in the days when you had to drop into ASCII to implement commands, or worked the last ten years of their career translating medical conditions into an automated program to apply medical codes to a document, could possibly understand anything about VTT....because, of course, they're old.
cd
Rollback Post to RevisionRollBack
To post a comment, please login or register a new account.
Today, I received an invitation to complete a survey for D&D Beyond...a site I find useful for gaming, easy to use, and was invaluable during the pandemic since our group, which plays several times a month, never had to miss a chance to play at our favorite hobby. I appreciate the site well enough to buy a subscription. So I was happy to agree to help out with survey questions.
The first question requested my age. After filling in the box, the survey 'thanked me' for participating and ended.
Was it because I, truthfully, admitted to being 76 years old?
I've been playing D&D for over forty years, from 2nd edition on up, as both player and DM. I've spent....probably thousands...on rule books, modules, conventions and dice. I've also play-tested many other games and keep coming back to this one....why? Because I love it!
I'm truly disappointed that D&D Beyond doesn't think my thoughts on the game and opinions matter anymore.
Sincerely,
Carol Dodd
That's a shame.
On the other hand I got bored at about 25% progress after endless personality and targeted marketing and demographic questions. I thought I was going to be able to give feedback on the site itself... maybe after another 15 minutes of marketing questions.
Surveys usually have quotas for how many replies they need of given demographics and it's much more likely that the age demographic you fall into had reached its quota. It's not that your opinions don't matter because of your age, it's just that surveys have limits on how many responses they look for.
Find my D&D Beyond articles here
The survey was heavily skewed and all about the VTT. For one section you HAD TO pick which option was the “most compelling,” even if you found none of the options compelling at all, and it asked what websites we frequently used, but didn’t include crowd favorites such as DM’s Guild and Hero Forge among the drop-down choices. It was a very badly designed survey.
Creating Epic Boons on DDB
DDB Buyers' Guide
Hardcovers, DDB & You
Content Troubleshooting
I received an invitation to a survey from D&D Beyond about a virtual table top that started off being annoying by claiming its contents are a secret, which is a completely unenforceable claim, then became more painful and time consuming until the questions continued repeating over and over in varying combinations and were un-skippable. At that point I decided the survey was outright harassment and closed it a little more than half way through. Whoever the social scientist is that designed the survey should be fired. Real people are not your Monte Carlo simulation tools.
This is the worst designed survey I have ever seen. That includes surveys that were so vague and ambiguous that the results were nearly meaningless. Harassing customers does not make them want to re-subscribe to D&D Beyond's services and almost convinced me to request cancellation and a prorated refund.
Terrible response blaming quotas. I’d like to see the “quota” for 65+, as I’m sure it’s 0 😡
It's how surveys operate; you break your survey down into demographic groups and then aim to get an equal number of responses per grouping. This gives you a more representative breakdown across demographics rather than just eschewed towards the dominant demographic.
You can make presumptions of malice if you wish, but I can assure you as someone who is married to a research doctor who conducts a lot of surveys and studies (and based her thesis on it), it's a normal survey approach.
There are lots of interesting survey tricks that people aren't aware of. For example, have you ever wondered by the same question may come up two or three times, but the answer key seems shuffled around? This is a confidence assessment tool. If someone is engaging with the survey genuinely (ie answering honestly and thoughtfully) the answers on those repeat questions will line up more often than not. However, if the person is just clicking through without really thinking, those same questions will have different answers more often than not. This can indicate how much confidence can be had with the survey data.
There are lots of little tools that are used to make the data as useable as possible beyond just collecting as much information as possible.
Find my D&D Beyond articles here
I made it into the 55-64 group but found the survey to be.. iffy at best.
outside of general "what do you plan and how do you play games, rpgs" and "website's you use" (a lot of marketing or weird way to break down later answers), it was all VTT. Rewatch the video we've seen on it and tell us what you thought - and some horribly done rankings: picking compelling or unique across 3 options.. which were really around 8 just in different groupings with no option to say "nothing is compelling nor unique".
Personally I still find the VTT gimmicky. a lot of pretty miniature images and spell graphics but they keep talking about 'simple' and, to be honest, even simplified 4e (horrible version) seems more complex than their VTT look. I keep getting the feel the dumbing down of familiars and druids (really one set of stats per but different looks) is all about trying to squeeze it into their VTT. I doubt it will do anything like Roll20, much less FGU given what they've shown us. Heck, they can't even get their character sheets to do non-standard attacks correctly (no divine strike for paladin, no selection on weapons with multiple damage options).
Yeah, I don't need a 3D VTT, and the video seemed more about advertising the 3D and graphics over the things the text was emphasizing, like rules.
Birgit | Shifter | Sorcerer | Dragonlords
Shayone | Hobgoblin | Sorcerer | Netherdeep
Thank you, all, for your replies!
I'm pretty well aware of how surveys operate. I mean gaming isn't all I've done for the last 40 years. And since I clicked on the survey invitation as soon as I saw an email had popped up in my multiple open windows, my conclusion is that the number of people in my particular 'demographic' was 0.
I would have loved to discuss ONE VTT TO RULE THEM ALL, considering how important and handy online rpg became during the pandemic. Right now, to play our campaigns, we are using a combo of three apps....D&D Beyond for character sheets, Roll 20 for game maps, and Discord, because Roll 20's audio is lousy. We've attempted to use Avrae for logging purposes, but about half of us can't seem to connect properly, in spite of the fact that our D&D Beyond accounts are linked to Discord.
At any rate, I appreciate all of your comments! Gamers are the best!
Carol
I hope your wife and others realise that using such "tricks" also poisons the well. If the survey asker doesn't compensate me for my time in some way (and DDB generally doesn't, not even like a freebie character sheet frame or something, and most surveys don't these days), then they're doing it on my generosity and on my charity. I'm willing to do that and generally put in a good faith effort, but when they're asking me the same question over and over...it just tells me that not only do they not value my time very much...but they're actively wasting it. On a good day, it'll put me off and I'll be less likely to engage with the content. I haven't done it yet, but catch me on a bad day and I'll start wasting their time in return.
The worst one I've had,.and I believe it was a DDB one, literally had entire sections repeated. Half the time I spent on that survey was repeating myself...sometimes several times. I still answered in good faith...but I've lost patience with it now.
If you're not willing or able to to discuss in good faith, then don't be surprised if I don't respond, there are better things in life for me to do than humour you. This signature is that response.
I had someone refer me to this entry. I was also dropped after posting my age.
The last 25 years I worked for a not-for-profit defense company that consisted of 80 scientists with “Doctor” in front of their names who did human research things like surveys.
About 1/3 of surveys were for national governments, state governments and individual companies across the planet. US clients include DOD, CIA, FBI, DEA, DOJ, DOT, Departments of Commerce, Interior, Veterans Affairs, and the list goes on. The surveys your kids and teachers get in High school have a good chance of being designed by that company (42 of the 50 states).
Survey update training by big name companies for collection methods and results and lessons learned from our surveys were weekly and monthly. Nothing enrages a client like angry survey respondents who would call their member of Congress over a survey we created for them.
I took an active part of survey creation, evaluation, distribution, response handling and data distribution using our own data, email and web servers until we went to using companies like Survey Monkey.
I mention this wall of text to point out I know surveys, from creation to collection to analysis of data.
It is a very poor survey that cuts off responses based on any factor on collection. The only valid way to do it is after data collection where it’s a simple toggle on an output but you still have the data for later analysis.
The fact they excluded based on age shows just how poorly the client understands the people who buy their products and how badly assumptions on the user population and data were built into the survey.
A VTT is not a video game and this survey may be an example of the “If you only have a hammer everything begins to look like a nail” problem when the background of those in charge comes from video games and not role-playing games. A constant refrain was “Check your perspective” when designing a survey.
Role playing gamers are different from video game players even if many users are in both populations.
Only 6 percent of video gamers are over 65 in 2022. This is an accurate hard number from places like Statista. In several sloppy online databases, who I will not name, this statistic is improperly labeled “Role Playing Gamers” Is it an accident the survey stops if you enter 65 or older?
The percentage of Role-Playing Gamers at 65 and older is actually 17% in 2022. This is roughly 1 out of every 6 players and almost 3 times that of video gamers. Considering the greater percentage of DM’s vs just players over 65 and the fact they purchase a much larger percentage of products for their groups shows Wizards should be checking their perspective.
A successful survey should understand its target audience and make sure they collect the correct data for the customer to make informed decisions on what they are trying to do. I think that was not accomplished by everything I have read and I bet the fault starts with incorrect consumer assumptions.
My second observation is that a VTT cannot compete with a video game and will only succeed with satisfied DM’s who let their groups use it and this survey dropped them when they entered an age past 65...
I think you just nailed it, RedSix!
I stopped answering after it started asking me personality type questions and questions about my anxiety levels...i know we live in a time where everyone wants everything about them online, but goddamn, no thanks.
Yeah, I agree. Our DM has been posting the rule changes proposed for the next edition and asking us to respond.
All seem designed to nullify or decrease any species or class advantages that are part of the existing rule set. My first question is always: "You're not planning to implement this in our games, are you?" Fortunately, his answer has always been "No."
I try not to make the same kind of age-biased comments about the young that are generally lodged against older people. But in my mind's ear I can hear complaints that begin in a whining tone and amount to: "But that's not fair....."
IMO, the only question that should be asked is; "How does this make the game more fun?"
In this case, I believe that they don't think anyone who worked as a Compuserve admin, back in the days when you had to drop into ASCII to implement commands, or worked the last ten years of their career translating medical conditions into an automated program to apply medical codes to a document, could possibly understand anything about VTT....because, of course, they're old.
cd