Must every survey for any Unearthed Arcana be 3 hours long? I have a job I get paid for. I don't want tax-level paperwork I don't get paid for.
Must it really ask me the same question three different times? That feels rude and inane. This shift into a waste of time and implies a need to hate D&D. This is a game, which means I should be increasing my fun, not my workload.
I have attempted to finsh every survey sent to me and even during my off hours when I think the hours are relaxed. NOT ONCE have I been able to finish one more than 70 percent.
LIFE gets int he way of these bureaucratic attampts to get me to say the right thing int he right way so you can claim my answer was what you were looking to do anyway. My incomplete is carried off the field as a win for the other side.
Inordinately long surveys are the kith and kin of those who have "no life responsibilities" so your surveys are skewed to the young and old and generaly negates the workforce paying for your products.
You don't have to do the survey all at once. So long as you don't close the tab, you can do some, do other stuff, and come back to the survey even after shutting down your computer for the night. Lets you do the survey over the course of several days, which is the only reason I've gotten through as many as I have.
The company also asks you what you want to rate for a reason. It doesn't expect everyone to rate everything, even if that assumption is annoying. They're looking to see what gets strong reactions; if your response to most of the content of a document is "meh", that's not really what they're trying to gauge.
But yes, both the documents and the surveys are way too goddamn long. I really miss the shorter, more frequent monthly drops.
You don't have to do the survey all at once. Do long as you don't close the tab, you can do some, fo other stuff, and come back to the survey even after shutting down your computer for the night. Lets you do the survey over the course of several days, which is the only reason I've gotten through as many as I have.
The company also asks you what you want to rate for a reason. It doesn't expect everyone to rate everything, even if that assumption is annoying. They're looking to see what gets strong reactions; if your response to most of the content of a document is "meh", that's not really what they're trying to gauge.
But yes, both the documents and the surveys are way too goddamn long. I really miss the shorter, more frequent monthly drops.
This only works if you're filling out the survey on a personal home computer. Those with us trying to do it on mobile or on shared PCs don't get this luxury.
It really should break each survey down into multiple pieces. Frankly I think each class should be a separate survey, because there are so many moving parts for each. I don't blame anyone who waits until guys like XP to Level 3, NerdImmersion, Treeantmonk, D4 Deep Dive etc do their survey filling streams because there's so much material to go through for classes you might not have as much personal experience with to make a fully informed comparison and those guys are literally getting paid to carefully study it. Everyone knows that the Berserkar Barbarian in 2014 is trash, but Totem Warrior wasn't available to everyone and not everybody can play Baldur's Gate 3 to get a sense of what it's doing.
Yeah, instead of having to select everything beforehand in a single survey, it would be much better to have an individual survey for each class & system mechanic (i.e., spells or weapon masteries). That way you have shorter surveys and can keep giving feedback later or if you playtest some more.
I don't know about the survey being too long, but if I was going to criticise it for anything it would be how the quantitative parts of the questionnaire ("Very Dissatisfied", "Dissatisfied", "Satisfied", "Very Satisfied") is a bit iffy from a survey design point of view since it forces a person to take a stance. There's no neutral opinion except not answering at all (and null is not the same as neutral). That will make sure the survey will always have a result showing a tendency to either direction of a mid-point. When designing a survey, you will always want to use an odd number of scoring options to give people the chance to say "I experienced this, but I was neither happy nor unhappy with it." Not doing that is a borderline methodological error in my opinion.
Also, I don't envy the people who have to go through the qualitative parts of the questionnaire (free-form entries), and divide them into thematically linked units for analysis. Ooof. That's a rough part about survey research!
... BUT!
More than anything, I just want them to switch the position of the Next and Back buttons. Come on!
I don't know about the survey being too long, but if I was going to criticise it for anything it would be how the quantitative parts of the questionnaire ("Very Dissatisfied", "Dissatisfied", "Satisfied", "Very Satisfied") is a bit iffy from a survey design point of view since it forces a person to take a stance. There's no neutral opinion except not answering at all (and null is not the same as neutral). That will make sure the survey will always have a result showing a tendency to either direction of a mid-point. When designing a survey, you will always want to use an odd number of scoring options to give people the chance to say "I experienced this, but I was neither happy nor unhappy with it." Not doing that is a borderline methodological error in my opinion.
Also, I don't envy the people who have to go through the qualitative parts of the questionnaire (free-form entries), and divide them into thematically linked units for analysis. Ooof. That's a rough part about survey research!
... BUT!
More than anything, I just want them to switch the position of the Next and Back buttons. Come on!
*shakes a tiny fist*
Whew. That felt good. Okay. I'm good now. :)
There is a reason for removing neutral options though. Many in research have pointed out (though there are two polarized schools of thought on this) that when a neutral option is available, it tends to be used even when the person surveyed may not truly feel neutral, leading to a response bias. I believe a truly neutral feeling is unlikely in a survey with no material incentive and which is purely taken because of a desire to contribute to the development of the game. By removing the neutral option from the likert scale, it forces people to, as you say, take a stance. One of my statistics profs always used to say 'if you drill down deep enough, no one feels neutral about anything'.
Must every survey for any Unearthed Arcana be 3 hours long? I have a job I get paid for. I don't want tax-level paperwork I don't get paid for.
Must it really ask me the same question three different times? That feels rude and inane. This shift into a waste of time and implies a need to hate D&D. This is a game, which means I should be increasing my fun, not my workload.
I have attempted to finsh every survey sent to me and even during my off hours when I think the hours are relaxed. NOT ONCE have I been able to finish one more than 70 percent.
LIFE gets int he way of these bureaucratic attampts to get me to say the right thing int he right way so you can claim my answer was what you were looking to do anyway. My incomplete is carried off the field as a win for the other side.
Inordinately long surveys are the kith and kin of those who have "no life responsibilities" so your surveys are skewed to the young and old and generaly negates the workforce paying for your products.
You don't have to do the survey all at once. So long as you don't close the tab, you can do some, do other stuff, and come back to the survey even after shutting down your computer for the night. Lets you do the survey over the course of several days, which is the only reason I've gotten through as many as I have.
The company also asks you what you want to rate for a reason. It doesn't expect everyone to rate everything, even if that assumption is annoying. They're looking to see what gets strong reactions; if your response to most of the content of a document is "meh", that's not really what they're trying to gauge.
But yes, both the documents and the surveys are way too goddamn long. I really miss the shorter, more frequent monthly drops.
Please do not contact or message me.
Whatever life holds in store for me, I will never forget these words: 'With great power comes great responsibility. ' This is my gift, my curse.
We have the power to do the surveys, long or short.
DM mostly, Player occasionally | Session 0 form | He/Him/They/Them
EXTENDED SIGNATURE!
Doctor/Published Scholar/Science and Healthcare Advocate/Critter/Trekkie/Gandalf with a Glock
Try DDB free: Free Rules (2024), premade PCs, adventures, one shots, encounters, SC, homebrew, more
Answers: physical books, purchases, and subbing.
Check out my life-changing
This only works if you're filling out the survey on a personal home computer. Those with us trying to do it on mobile or on shared PCs don't get this luxury.
It really should break each survey down into multiple pieces. Frankly I think each class should be a separate survey, because there are so many moving parts for each. I don't blame anyone who waits until guys like XP to Level 3, NerdImmersion, Treeantmonk, D4 Deep Dive etc do their survey filling streams because there's so much material to go through for classes you might not have as much personal experience with to make a fully informed comparison and those guys are literally getting paid to carefully study it. Everyone knows that the Berserkar Barbarian in 2014 is trash, but Totem Warrior wasn't available to everyone and not everybody can play Baldur's Gate 3 to get a sense of what it's doing.
Yeah, instead of having to select everything beforehand in a single survey, it would be much better to have an individual survey for each class & system mechanic (i.e., spells or weapon masteries). That way you have shorter surveys and can keep giving feedback later or if you playtest some more.
I don't know about the survey being too long, but if I was going to criticise it for anything it would be how the quantitative parts of the questionnaire ("Very Dissatisfied", "Dissatisfied", "Satisfied", "Very Satisfied") is a bit iffy from a survey design point of view since it forces a person to take a stance. There's no neutral opinion except not answering at all (and null is not the same as neutral). That will make sure the survey will always have a result showing a tendency to either direction of a mid-point. When designing a survey, you will always want to use an odd number of scoring options to give people the chance to say "I experienced this, but I was neither happy nor unhappy with it." Not doing that is a borderline methodological error in my opinion.
Also, I don't envy the people who have to go through the qualitative parts of the questionnaire (free-form entries), and divide them into thematically linked units for analysis. Ooof. That's a rough part about survey research!
... BUT!
More than anything, I just want them to switch the position of the Next and Back buttons. Come on!
*shakes a tiny fist*
Whew. That felt good. Okay. I'm good now. :)
There is a reason for removing neutral options though. Many in research have pointed out (though there are two polarized schools of thought on this) that when a neutral option is available, it tends to be used even when the person surveyed may not truly feel neutral, leading to a response bias. I believe a truly neutral feeling is unlikely in a survey with no material incentive and which is purely taken because of a desire to contribute to the development of the game. By removing the neutral option from the likert scale, it forces people to, as you say, take a stance. One of my statistics profs always used to say 'if you drill down deep enough, no one feels neutral about anything'.
DM mostly, Player occasionally | Session 0 form | He/Him/They/Them
EXTENDED SIGNATURE!
Doctor/Published Scholar/Science and Healthcare Advocate/Critter/Trekkie/Gandalf with a Glock
Try DDB free: Free Rules (2024), premade PCs, adventures, one shots, encounters, SC, homebrew, more
Answers: physical books, purchases, and subbing.
Check out my life-changing
Agree, listening to the screen reader for that long is too long. Keep it simple and keep the places for people to write comments.
And then read the comments if you want to know what we think.
I didn’t see what you did there.