Not sure if this is the right place to post feedback about the current survey...
But wow, really badly designed! Long winded, doesn't take previous answers into consideration so you end up answering things twice and took me nearly 15 minutes to complete.
I got to the point where I just stopped answering the question if I didn't think it was important just to get through it! Plus asking how much people earn should be irrelevant.
I do think that it's important for WotC and DDB to understand the people who play, and how they play but this will just put people off :/
People's earnings are relevant because how much they earn practically defines what part of the market they belong to and what they should be aiming for.
If they have a ton of people on low income and very few people on high income playing, then they really need the target low income consumers. Make books as cheap as possible, low thrills, no extras etc, so they can get the price as low as possible, making it accessible to all those people who don't have much money to spare. If they could get the books down to, say, $20 each, then they'd sell to a load of them and make profits through sheer numbers.
On the other hand, if almost every player is a multimillionaire, then they'd never think twice about splashing out $300 on a nice book, so they could focus more on books like Beadle & The Bard series, or ones made of gold or something.
Those are extreme examples but they illustrate the point. Knowing the pricing point of what you can sell is one of the most important bits of information for a business.
Was it long? It did feel it. I agree that if you answer a question, then it should automatically answer the same question later on (I wonder if that's because they cheaped out on the company that did it - it's extra effort and money to do that kind of programming). I'm also of a mind to stop responding to them because the surveys are long and I don't particularly find the questions interesting.
Perhaps they could offer free dice or something for filling them out.
Rollback Post to RevisionRollBack
If you're not willing or able to to discuss in good faith, then don't be surprised if I don't respond, there are better things in life for me to do than humour you. This signature is that response.
doesn't take previous answers into consideration so you end up answering things twice
This is actually an intentional design feature of many surveys intended to provide data on the confidence level of responses. Sometimes people will just breeze through a survey without actually paying attention, clicking random answers for whatever reason. This is not good data and is a waste of time to process.
So what you do is include the same (or similar) questions multiple times, usually with the answer key shuffled around. If someone is actually engaging with the survey, these answers will match and you'll have a high confidence in the data. If someone is however just clicking through without paying attention, there'll be a higher discrepancy and the confidence in the data will be lower.
This can even be incorporated into a weighting system where the answers from surveys with a high confidence are given more importance than those with low confidence. It can be as simple as each question being worth a number of points equal to the confidence percentage of the overall survey.
Going off of what Davyd said - the point of a survey is to gather good data and feedback. If you get annoyed and skip questions or ignore whole parts of the survey, then you may as well not have done it because the data you submit will be wildly incomplete/inaccurate.
Rollback Post to RevisionRollBack
I know what you're thinking: "In that flurry of blows, did he use all his ki points, or save one?" Well, are ya feeling lucky, punk?
doesn't take previous answers into consideration so you end up answering things twice
This is actually an intentional design feature of many surveys intended to provide data on the confidence level of responses. Sometimes people will just breeze through a survey without actually paying attention, clicking random answers for whatever reason. This is not good data and is a waste of time to process.
So what you do is include the same (or similar) questions multiple times, usually with the answer key shuffled around. If someone is actually engaging with the survey, these answers will match and you'll have a high confidence in the data. If someone is however just clicking through without paying attention, there'll be a higher discrepancy and the confidence in the data will be lower.
This can even be incorporated into a weighting system where the answers from surveys with a high confidence are given more importance than those with low confidence. It can be as simple as each question being worth a number of points equal to the confidence percentage of the overall survey.
Someone took "research methods" in college.
A very crucial and important part of creating surveys. This is akin to making a "control group" for when you are doing experiments.
Going off of what Davyd said - the point of a survey is to gather good data and feedback. If you get annoyed and skip questions or ignore whole parts of the survey, then you may as well not have done it because the data you submit will be wildly incomplete/inaccurate.
Which is why the survey creators need to ensure that their surveys aren't longer than necessary. When they're wanting you to spend 25 minutes on it and it's mostly asking about how much you've spent on specific categories...it is just more frustrating when you realise they're spending your time making you repeat yourself. It wasn't exactly the 1DND surveys where at least you're directly contributing to the game.
Rollback Post to RevisionRollBack
If you're not willing or able to to discuss in good faith, then don't be surprised if I don't respond, there are better things in life for me to do than humour you. This signature is that response.
doesn't take previous answers into consideration so you end up answering things twice
This is actually an intentional design feature of many surveys intended to provide data on the confidence level of responses. Sometimes people will just breeze through a survey without actually paying attention, clicking random answers for whatever reason. This is not good data and is a waste of time to process.
So what you do is include the same (or similar) questions multiple times, usually with the answer key shuffled around. If someone is actually engaging with the survey, these answers will match and you'll have a high confidence in the data. If someone is however just clicking through without paying attention, there'll be a higher discrepancy and the confidence in the data will be lower.
This can even be incorporated into a weighting system where the answers from surveys with a high confidence are given more importance than those with low confidence. It can be as simple as each question being worth a number of points equal to the confidence percentage of the overall survey.
Building on the above, it should be noted that this survey is full of dependencies. For example, if you click "I have only played 4th Edition" it will ask you questions about why you have not switched over to 5th Edition. I only played around with that part of the survey to test, but I expect there are other questions dependent on answers to others in the survey as well.
Dependent questions are great for making sure survey respondents do not have to constantly click "not applicable" or otherwise spend time disposing of questions irrelevant to them. The downside is that certain important questions might get tied up in a dependency chain and could possibly be missed. Having the same or similar questions in multiple locations allows you to better streamline your dependent questions while still ensuring that the respondent is likely to see and answer any given question.
It wasn't exactly the 1DND surveys where at least you're directly contributing to the game.
It should be noted that these surveys do contribute greatly and directly to the game. Questions about what settings you like, for example, drive major decisions in terms of what products we get (Spelljammer, for example, was brought back due to surveys like this), and responses showing things like "Druid is the least popular class" directly inform how those subjects are developed moving forward.
A very crucial and important part of creating surveys. This is akin to making a "control group" for when you are doing experiments.
Many are those eager to participate in surveys and engage purely to influence data and not always for the most honorable reasons. Some might be sincere in their responses but simply grow weary of repeating themselves and if this shows then have their perfectly well-intentioned responses disregarded. Unless you're just surveying people's patience I really don't think this 'research method' is giving you good data. That's not even to touch upon the fact the 'control group' in this context isn't likely to be taken from a broad cross-section of players of the game who play differently but really just those who use Beyond to do so. That's a bit like a company only surveying those customers who shop the way they want them to. How is that even remotely reliable?
INSPIRATIONS:Clark Ashton Smith, Mervyn Peake, Jack Vance, Michael Moorcock, Fritz Leiber, M. John Harrison, Gene Wolfe, Steven Brust, Terry Pratchett, China Miéville.
doesn't take previous answers into consideration so you end up answering things twice
This is actually an intentional design feature of many surveys intended to provide data on the confidence level of responses. Sometimes people will just breeze through a survey without actually paying attention, clicking random answers for whatever reason. This is not good data and is a waste of time to process.
So what you do is include the same (or similar) questions multiple times, usually with the answer key shuffled around. If someone is actually engaging with the survey, these answers will match and you'll have a high confidence in the data. If someone is however just clicking through without paying attention, there'll be a higher discrepancy and the confidence in the data will be lower.
This can even be incorporated into a weighting system where the answers from surveys with a high confidence are given more importance than those with low confidence. It can be as simple as each question being worth a number of points equal to the confidence percentage of the overall survey.
Someone took "research methods" in college.
A very crucial and important part of creating surveys. This is akin to making a "control group" for when you are doing experiments.
Actually my wife is a doctor who uses surveys as part of her research methodologies, both quantitatively and qualitatively
Actually my wife is a doctor who uses surveys as part of her research methodologies, both quantitatively and qualitatively
A survey intended to see what players of D&D think really only surveying those who use D&D Beyond is a bit like a survey intended to see what readers think really only surveying those who buy books using a credit card and from the same online bookstore. That is a methodology that is deeply flawed unless a company just wants to see what customers who shop how they want them to shop are thinking.
Rollback Post to RevisionRollBack
INSPIRATIONS:Clark Ashton Smith, Mervyn Peake, Jack Vance, Michael Moorcock, Fritz Leiber, M. John Harrison, Gene Wolfe, Steven Brust, Terry Pratchett, China Miéville.
That is another way for the surveyors to pick out those with strong opinions.
Just like junk e-mails/texts have spelling mistakes in them on purpose, since folks who are likely to spot spelling mistakes are less likely to fall for the con.
That is another way for the surveyors to pick out those with strong opinions.
Just like junk e-mails/texts have spelling mistakes in them on purpose, since folks who are likely to spot spelling mistakes are less likely to fall for the con.
The thing is, that method would work for the 1D&D playtests, but when a lot of the survey was "How much did you spend on the official figures in the last three months?"... it's less meaningful. Very few people will be dying to share their "opinion" on that.
Rollback Post to RevisionRollBack
If you're not willing or able to to discuss in good faith, then don't be surprised if I don't respond, there are better things in life for me to do than humour you. This signature is that response.
Actually my wife is a doctor who uses surveys as part of her research methodologies, both quantitatively and qualitatively
A survey intended to see what players of D&D think really only surveying those who use D&D Beyond is a bit like a survey intended to see what readers think really only surveying those who buy books using a credit card and from the same online bookstore. That is a methodology that is deeply flawed unless a company just wants to see what customers who shop how they want them to shop are thinking.
Well, as someone who does qualitative and quantitative research for academic, governmental, and for-profit businesses, I can note that this statement is predicated on some presumptions that cannot be known.
1 - It could easily be the intent to only survey DDB users.
2 -- The survey could be being used in other places, but we don't know.
3 -- That only surveying what "those who buy books using a credit card and from the same online bookstore" has no value to the company doing the research (I can think of a half dozen possible reasons). If the goals and the queries are limited to those factors, then there is no flaw in the methodology, because it is designed to do exactly what they needed to collect.
4 -- One needs to know the purpose of the survey and the usage of the dataset in order to determine if there are any real flaws -- even in your "unless" example. Since we don't know what the purpose is and do not likely have access to see the use of the data set, we cannot make any conclusions about methodology because we do not have the information available for such a conclusion.
That said, I can draw some inferences and suppositions based on my experience in development of, use of, and analysis of such data, and from that come up with some vague, hand waveable, somewhat informed guesses, but they wouldn't be of any real use, merely fun speculation.
I would say that the likelihood is that they are collecting data to see and model what potential uptake of internal proposals regarding marketplace and product assortment among "highly Engaged" users, so they can determine better how to structure and price things as they migrate existing spaces and tools to this site from others, as well as possibly identify additional distribution channels or identify ways to refine marketing and outreach.
Match against previous data (both direct request and ancillary collection) and you can form a very reliable large scale predictive model and generate some nice scattergraphs that show likelihood that can then be broken down using traditional toolsets for more effectively targeted campaigns that make use of the massive data sets available already regarding populations (many of which do measure such things as what is the chance they play D&D).
It is a good, standard, reliable survey base, and the only possible detractions one could take on it would only apply if it was going to be used for purposes other than business development, in which case the issue is the relatively minor and easily controlled for self selection bias aspect. But for the kind of survey it was and the statistically likely use to which it will be put, it was a fairly blah version of a standard set up that is used in about 90% of for profit industry.
I initially cleared it thinking it was just more stuff about the book or about the movie, but I caught "survey as I was clicking and ended up going back to click it and took it. Wasn't very long, was pretty basic, and since I am an outlier, I expect my stuff won't have great impact.
But who knows, maybe they will put out more stuff for DMs and make VTT stuff more useful for the heavy customizers.
Rollback Post to RevisionRollBack
Only a DM since 1980 (3000+ Sessions) / PhD, MS, MA / Mixed, Bi, Trans, Woman / No longer welcome in the US, apparently
Wyrlde: Adventures in the Seven Cities .-=] Lore Book | Patreon | Wyrlde YT [=-. An original Setting for 5e, a whole solar system of adventure. Ongoing updates, exclusies, more. Not Talking About It / Dubbed The Oracle in the Cult of Mythology Nerds
Well, as someone who does qualitative and quantitative research for academic, governmental, and for-profit businesses, I can note that this statement is predicated on some presumptions that cannot be known.
1 - It could easily be the intent to only survey DDB users.
2 -- The survey could be being used in other places, but we don't know.
3 -- That only surveying what "those who buy books using a credit card and from the same online bookstore" has no value to the company doing the research (I can think of a half dozen possible reasons). If the goals and the queries are limited to those factors, then there is no flaw in the methodology, because it is designed to do exactly what they needed to collect.
4 -- One needs to know the purpose of the survey and the usage of the dataset in order to determine if there are any real flaws -- even in your "unless" example. Since we don't know what the purpose is and do not likely have access to see the use of the data set, we cannot make any conclusions about methodology because we do not have the information available for such a conclusion.
That said, I can draw some inferences and suppositions based on my experience in development of, use of, and analysis of such data, and from that come up with some vague, hand waveable, somewhat informed guesses, but they wouldn't be of any real use, merely fun speculation.
I would say that the likelihood is that they are collecting data to see and model what potential uptake of internal proposals regarding marketplace and product assortment among "highly Engaged" users, so they can determine better how to structure and price things as they migrate existing spaces and tools to this site from others, as well as possibly identify additional distribution channels or identify ways to refine marketing and outreach.
Match against previous data (both direct request and ancillary collection) and you can form a very reliable large scale predictive model and generate some nice scattergraphs that show likelihood that can then be broken down using traditional toolsets for more effectively targeted campaigns that make use of the massive data sets available already regarding populations (many of which do measure such things as what is the chance they play D&D).
It is a good, standard, reliable survey base, and the only possible detractions one could take on it would only apply if it was going to be used for purposes other than business development, in which case the issue is the relatively minor and easily controlled for self selection bias aspect. But for the kind of survey it was and the statistically likely use to which it will be put, it was a fairly blah version of a standard set up that is used in about 90% of for profit industry.
I initially cleared it thinking it was just more stuff about the book or about the movie, but I caught "survey as I was clicking and ended up going back to click it and took it. Wasn't very long, was pretty basic, and since I am an outlier, I expect my stuff won't have great impact.
But who knows, maybe they will put out more stuff for DMs and make VTT stuff more useful for the heavy customizers.
More stuff for DMs would be nice.
Those presumptions need not be presumed if they were just frank with us and told us if they're only really interested in knowing what it is those who use Beyond (instead of a broader cross-section of players) want out of D&D. And why that is if that is true.
I've seen more than my fair share of unfair criticisms of the methodologies behind certain surveys and polls and forms from those with no real knowledge of their purposes or from those who even insisted their purposes were the opposite of what they were to see where you're coming from. But I've little faith in anyone being immune to this. Myself included. Those of us with strong opinions are funny like that.
Rollback Post to RevisionRollBack
INSPIRATIONS:Clark Ashton Smith, Mervyn Peake, Jack Vance, Michael Moorcock, Fritz Leiber, M. John Harrison, Gene Wolfe, Steven Brust, Terry Pratchett, China Miéville.
Actually my wife is a doctor who uses surveys as part of her research methodologies, both quantitatively and qualitatively
A survey intended to see what players of D&D think really only surveying those who use D&D Beyond is a bit like a survey intended to see what readers think really only surveying those who buy books using a credit card and from the same online bookstore. That is a methodology that is deeply flawed unless a company just wants to see what customers who shop how they want them to shop are thinking.
I don't agree that it's flawed. If you want to capture data from people who play D&D, a site that caters exclusively to people that play D&D would be the best place to sample from.
Also there's the practical limitation; they can't install their survey prompt on other people's sites, it's not like irl surveying where you can stand outside a store or on the street
I don't agree that it's flawed. If you want to capture data from people who play D&D, a site that caters exclusively to people that play D&D would be the best place to sample from.
Also there's the practical limitation; they can't install their survey prompt on other people's sites, it's not like irl surveying where you can stand outside a store or on the street
There is every possibility that players who use Beyond have disparate views to those who do not. Both are players of D&D. But they are very different types of consumers.
It's like someone who buys books exclusively from an online bookstore or who will only read e-books and someone who only buys physical books from brick-and-mortal stores. You can't expect the results from surveys targeting only the first of these to reflect the views of the general reading population.
You're right about the practical limitation. But here I am on Beyond and seeing this thread is the first I'm hearing about this survey.
Rollback Post to RevisionRollBack
INSPIRATIONS:Clark Ashton Smith, Mervyn Peake, Jack Vance, Michael Moorcock, Fritz Leiber, M. John Harrison, Gene Wolfe, Steven Brust, Terry Pratchett, China Miéville.
interesting for non-english D&D players, the survey (at least in german) lists localized products and also may show a local website instead of the usual english websites on some of the questions.
I guess they want to see how much of the localized stuff already out there is in use and where to spend more time and money to further the localizations since it is now all in house.
A survey intended to see what players of D&D think really only surveying those who use D&D Beyond is a bit like a survey intended to see what readers think really only surveying those who buy books using a credit card and from the same online bookstore. That is a methodology that is deeply flawed unless a company just wants to see what customers who shop how they want them to shop are thinking.
The survey doesn't only sample from users on D&D Beyond, I'm sure there are tons of people from Reddit or other discussion websites that take the survey. It doesn't even require an account here to participate if I remember correctly, and people that typically use different websites are fully capable of navigating to this site and/or the survey.
The way to effectively conduct a survey like this is very limited, because it isn't really possible to easily reach out to game players who aren't heavily involved Online, though 90% of players like that could give their feedback if they knew about the survey and wanted to participate. Yes, it would be great if local game stores and other physical places promoted and mentioned the survey, but it already reaches a large audience and there will always be word-of-the-mouth fans who help spread the news.
Also, Davyd raises an important point: Wizards can't easily just put or promote their survey on another entity's site, so Beyond is really the best place for this to be located.
Rollback Post to RevisionRollBack
BoringBard's long and tedious posts somehow manage to enrapture audiences. How? Because he used Charm Person, the #1 bard spell!
He/him pronouns. Call me Bard. PROUD NERD!
Ever wanted to talk about your parties' worst mistakes? Do so HERE. What's your favorite class, why? Share & explainHERE.
Going off of what Davyd said - the point of a survey is to gather good data and feedback. If you get annoyed and skip questions or ignore whole parts of the survey, then you may as well not have done it because the data you submit will be wildly incomplete/inaccurate.
Which is why the survey creators need to ensure that their surveys aren't longer than necessary. When they're wanting you to spend 25 minutes on it and it's mostly asking about how much you've spent on specific categories...it is just more frustrating when you realise they're spending your time making you repeat yourself. It wasn't exactly the 1DND surveys where at least you're directly contributing to the game.
If you look at the Species question the species listed are Human, Elf, Dwarf, Halfling, Gnome, Orc, Tiefling, Dragonborn, and Goliath, all playtested 1D&D species, but also 3 other species, Aasimar, Genasi, and Changelings. Why thoss one? Because I believe those three are goung to be added to the playtest, and they are testing for interest.
They also fit a pattern, Classic Tolkien races Human, Elf, Dwarf, Halfling, Gnome, Orc vs hybrid monster races Tiefling, Dragonborn, and Goliath, Aasimar, Genasi, and Changeling.
Rollback Post to RevisionRollBack
To post a comment, please login or register a new account.
Not sure if this is the right place to post feedback about the current survey...
But wow, really badly designed! Long winded, doesn't take previous answers into consideration so you end up answering things twice and took me nearly 15 minutes to complete.
I got to the point where I just stopped answering the question if I didn't think it was important just to get through it! Plus asking how much people earn should be irrelevant.
I do think that it's important for WotC and DDB to understand the people who play, and how they play but this will just put people off :/
People's earnings are relevant because how much they earn practically defines what part of the market they belong to and what they should be aiming for.
If they have a ton of people on low income and very few people on high income playing, then they really need the target low income consumers. Make books as cheap as possible, low thrills, no extras etc, so they can get the price as low as possible, making it accessible to all those people who don't have much money to spare. If they could get the books down to, say, $20 each, then they'd sell to a load of them and make profits through sheer numbers.
On the other hand, if almost every player is a multimillionaire, then they'd never think twice about splashing out $300 on a nice book, so they could focus more on books like Beadle & The Bard series, or ones made of gold or something.
Those are extreme examples but they illustrate the point. Knowing the pricing point of what you can sell is one of the most important bits of information for a business.
Was it long? It did feel it. I agree that if you answer a question, then it should automatically answer the same question later on (I wonder if that's because they cheaped out on the company that did it - it's extra effort and money to do that kind of programming). I'm also of a mind to stop responding to them because the surveys are long and I don't particularly find the questions interesting.
Perhaps they could offer free dice or something for filling them out.
If you're not willing or able to to discuss in good faith, then don't be surprised if I don't respond, there are better things in life for me to do than humour you. This signature is that response.
This is actually an intentional design feature of many surveys intended to provide data on the confidence level of responses. Sometimes people will just breeze through a survey without actually paying attention, clicking random answers for whatever reason. This is not good data and is a waste of time to process.
So what you do is include the same (or similar) questions multiple times, usually with the answer key shuffled around. If someone is actually engaging with the survey, these answers will match and you'll have a high confidence in the data. If someone is however just clicking through without paying attention, there'll be a higher discrepancy and the confidence in the data will be lower.
This can even be incorporated into a weighting system where the answers from surveys with a high confidence are given more importance than those with low confidence. It can be as simple as each question being worth a number of points equal to the confidence percentage of the overall survey.
Find my D&D Beyond articles here
Going off of what Davyd said - the point of a survey is to gather good data and feedback. If you get annoyed and skip questions or ignore whole parts of the survey, then you may as well not have done it because the data you submit will be wildly incomplete/inaccurate.
I know what you're thinking: "In that flurry of blows, did he use all his ki points, or save one?" Well, are ya feeling lucky, punk?
Someone took "research methods" in college.
A very crucial and important part of creating surveys. This is akin to making a "control group" for when you are doing experiments.
Blank
Which is why the survey creators need to ensure that their surveys aren't longer than necessary. When they're wanting you to spend 25 minutes on it and it's mostly asking about how much you've spent on specific categories...it is just more frustrating when you realise they're spending your time making you repeat yourself. It wasn't exactly the 1DND surveys where at least you're directly contributing to the game.
If you're not willing or able to to discuss in good faith, then don't be surprised if I don't respond, there are better things in life for me to do than humour you. This signature is that response.
Building on the above, it should be noted that this survey is full of dependencies. For example, if you click "I have only played 4th Edition" it will ask you questions about why you have not switched over to 5th Edition. I only played around with that part of the survey to test, but I expect there are other questions dependent on answers to others in the survey as well.
Dependent questions are great for making sure survey respondents do not have to constantly click "not applicable" or otherwise spend time disposing of questions irrelevant to them. The downside is that certain important questions might get tied up in a dependency chain and could possibly be missed. Having the same or similar questions in multiple locations allows you to better streamline your dependent questions while still ensuring that the respondent is likely to see and answer any given question.
It should be noted that these surveys do contribute greatly and directly to the game. Questions about what settings you like, for example, drive major decisions in terms of what products we get (Spelljammer, for example, was brought back due to surveys like this), and responses showing things like "Druid is the least popular class" directly inform how those subjects are developed moving forward.
Many are those eager to participate in surveys and engage purely to influence data and not always for the most honorable reasons. Some might be sincere in their responses but simply grow weary of repeating themselves and if this shows then have their perfectly well-intentioned responses disregarded. Unless you're just surveying people's patience I really don't think this 'research method' is giving you good data. That's not even to touch upon the fact the 'control group' in this context isn't likely to be taken from a broad cross-section of players of the game who play differently but really just those who use Beyond to do so. That's a bit like a company only surveying those customers who shop the way they want them to. How is that even remotely reliable?
INSPIRATIONS: Clark Ashton Smith, Mervyn Peake, Jack Vance, Michael Moorcock, Fritz Leiber, M. John Harrison, Gene Wolfe, Steven Brust, Terry Pratchett, China Miéville.
SYSTEMS: ShadowDark, C&C, AD&D.
GEAR: pencils, graph paper, dice.
Actually my wife is a doctor who uses surveys as part of her research methodologies, both quantitatively and qualitatively
Find my D&D Beyond articles here
Yes. Way too long. Gave up before half way.
A survey intended to see what players of D&D think really only surveying those who use D&D Beyond is a bit like a survey intended to see what readers think really only surveying those who buy books using a credit card and from the same online bookstore. That is a methodology that is deeply flawed unless a company just wants to see what customers who shop how they want them to shop are thinking.
INSPIRATIONS: Clark Ashton Smith, Mervyn Peake, Jack Vance, Michael Moorcock, Fritz Leiber, M. John Harrison, Gene Wolfe, Steven Brust, Terry Pratchett, China Miéville.
SYSTEMS: ShadowDark, C&C, AD&D.
GEAR: pencils, graph paper, dice.
That is another way for the surveyors to pick out those with strong opinions.
Just like junk e-mails/texts have spelling mistakes in them on purpose, since folks who are likely to spot spelling mistakes are less likely to fall for the con.
The thing is, that method would work for the 1D&D playtests, but when a lot of the survey was "How much did you spend on the official figures in the last three months?"... it's less meaningful. Very few people will be dying to share their "opinion" on that.
If you're not willing or able to to discuss in good faith, then don't be surprised if I don't respond, there are better things in life for me to do than humour you. This signature is that response.
Well, as someone who does qualitative and quantitative research for academic, governmental, and for-profit businesses, I can note that this statement is predicated on some presumptions that cannot be known.
1 - It could easily be the intent to only survey DDB users.
2 -- The survey could be being used in other places, but we don't know.
3 -- That only surveying what "those who buy books using a credit card and from the same online bookstore" has no value to the company doing the research (I can think of a half dozen possible reasons). If the goals and the queries are limited to those factors, then there is no flaw in the methodology, because it is designed to do exactly what they needed to collect.
4 -- One needs to know the purpose of the survey and the usage of the dataset in order to determine if there are any real flaws -- even in your "unless" example. Since we don't know what the purpose is and do not likely have access to see the use of the data set, we cannot make any conclusions about methodology because we do not have the information available for such a conclusion.
That said, I can draw some inferences and suppositions based on my experience in development of, use of, and analysis of such data, and from that come up with some vague, hand waveable, somewhat informed guesses, but they wouldn't be of any real use, merely fun speculation.
I would say that the likelihood is that they are collecting data to see and model what potential uptake of internal proposals regarding marketplace and product assortment among "highly Engaged" users, so they can determine better how to structure and price things as they migrate existing spaces and tools to this site from others, as well as possibly identify additional distribution channels or identify ways to refine marketing and outreach.
Match against previous data (both direct request and ancillary collection) and you can form a very reliable large scale predictive model and generate some nice scattergraphs that show likelihood that can then be broken down using traditional toolsets for more effectively targeted campaigns that make use of the massive data sets available already regarding populations (many of which do measure such things as what is the chance they play D&D).
It is a good, standard, reliable survey base, and the only possible detractions one could take on it would only apply if it was going to be used for purposes other than business development, in which case the issue is the relatively minor and easily controlled for self selection bias aspect. But for the kind of survey it was and the statistically likely use to which it will be put, it was a fairly blah version of a standard set up that is used in about 90% of for profit industry.
I initially cleared it thinking it was just more stuff about the book or about the movie, but I caught "survey as I was clicking and ended up going back to click it and took it. Wasn't very long, was pretty basic, and since I am an outlier, I expect my stuff won't have great impact.
But who knows, maybe they will put out more stuff for DMs and make VTT stuff more useful for the heavy customizers.
Only a DM since 1980 (3000+ Sessions) / PhD, MS, MA / Mixed, Bi, Trans, Woman / No longer welcome in the US, apparently
Wyrlde: Adventures in the Seven Cities
.-=] Lore Book | Patreon | Wyrlde YT [=-.
An original Setting for 5e, a whole solar system of adventure. Ongoing updates, exclusies, more.
Not Talking About It / Dubbed The Oracle in the Cult of Mythology Nerds
More stuff for DMs would be nice.
Those presumptions need not be presumed if they were just frank with us and told us if they're only really interested in knowing what it is those who use Beyond (instead of a broader cross-section of players) want out of D&D. And why that is if that is true.
I've seen more than my fair share of unfair criticisms of the methodologies behind certain surveys and polls and forms from those with no real knowledge of their purposes or from those who even insisted their purposes were the opposite of what they were to see where you're coming from. But I've little faith in anyone being immune to this. Myself included. Those of us with strong opinions are funny like that.
INSPIRATIONS: Clark Ashton Smith, Mervyn Peake, Jack Vance, Michael Moorcock, Fritz Leiber, M. John Harrison, Gene Wolfe, Steven Brust, Terry Pratchett, China Miéville.
SYSTEMS: ShadowDark, C&C, AD&D.
GEAR: pencils, graph paper, dice.
I don't agree that it's flawed. If you want to capture data from people who play D&D, a site that caters exclusively to people that play D&D would be the best place to sample from.
Also there's the practical limitation; they can't install their survey prompt on other people's sites, it's not like irl surveying where you can stand outside a store or on the street
Find my D&D Beyond articles here
There is every possibility that players who use Beyond have disparate views to those who do not. Both are players of D&D. But they are very different types of consumers.
It's like someone who buys books exclusively from an online bookstore or who will only read e-books and someone who only buys physical books from brick-and-mortal stores. You can't expect the results from surveys targeting only the first of these to reflect the views of the general reading population.
You're right about the practical limitation. But here I am on Beyond and seeing this thread is the first I'm hearing about this survey.
INSPIRATIONS: Clark Ashton Smith, Mervyn Peake, Jack Vance, Michael Moorcock, Fritz Leiber, M. John Harrison, Gene Wolfe, Steven Brust, Terry Pratchett, China Miéville.
SYSTEMS: ShadowDark, C&C, AD&D.
GEAR: pencils, graph paper, dice.
interesting for non-english D&D players, the survey (at least in german) lists localized products and also may show a local website instead of the usual english websites on some of the questions.
I guess they want to see how much of the localized stuff already out there is in use and where to spend more time and money to further the localizations since it is now all in house.
The survey doesn't only sample from users on D&D Beyond, I'm sure there are tons of people from Reddit or other discussion websites that take the survey. It doesn't even require an account here to participate if I remember correctly, and people that typically use different websites are fully capable of navigating to this site and/or the survey.
The way to effectively conduct a survey like this is very limited, because it isn't really possible to easily reach out to game players who aren't heavily involved Online, though 90% of players like that could give their feedback if they knew about the survey and wanted to participate. Yes, it would be great if local game stores and other physical places promoted and mentioned the survey, but it already reaches a large audience and there will always be word-of-the-mouth fans who help spread the news.
Also, Davyd raises an important point: Wizards can't easily just put or promote their survey on another entity's site, so Beyond is really the best place for this to be located.
BoringBard's long and tedious posts somehow manage to enrapture audiences. How? Because he used Charm Person, the #1 bard spell!
He/him pronouns. Call me Bard. PROUD NERD!
Ever wanted to talk about your parties' worst mistakes? Do so HERE. What's your favorite class, why? Share & explain
HERE.If you look at the Species question the species listed are Human, Elf, Dwarf, Halfling, Gnome, Orc, Tiefling, Dragonborn, and Goliath, all playtested 1D&D species, but also 3 other species, Aasimar, Genasi, and Changelings. Why thoss one? Because I believe those three are goung to be added to the playtest, and they are testing for interest.
They also fit a pattern, Classic Tolkien races Human, Elf, Dwarf, Halfling, Gnome, Orc vs hybrid monster races Tiefling, Dragonborn, and Goliath, Aasimar, Genasi, and Changeling.