That's true....hadn't thought of that Pocketmouse. I always wondered how well those free text data conversion tools/software work? Can they really get the nuances of context with them? I'm just used to dealing with continuous datasets, I suppose.
They're not going to carefully analyze thousands of comments, but it would be technically trivial to assemble all the comments from a particular page into a single large document and then read through it looking for common themes. That will probably miss comments that aren't made by a lot of people, but, well, comments that aren't repeated probably aren't very important. They could add things like word clouds to do more sophisticated text analysis, but it's not really needed.
They're not going to carefully analyze thousands of comments, but it would be technically trivial to assemble all the comments from a particular page into a single large document and then read through it looking for common themes. That will probably miss comments that aren't made by a lot of people, but, well, comments that aren't repeated probably aren't very important. They could add things like word clouds to do more sophisticated text analysis, but it's not really needed.
Yep. That makes sense too....
Actually, it would be kinda cool if they released a report someday that shows how they analyzed feedback and what the results were. If they've done this before, I've never seen it? It would also only appeal to some folks, I suppose. Still, would be interesting!
As an aside, I figure the more concise your comments are, the more likely someone is to actually read and process them. I can envision someone's eyes glazing over at a comment box filled with tons and tons of text.
To use the written comments as "data" they'd have to somehow convert those to a numerical system that can be analyzed. (Imagine trying to convert the massive volume of written comments to some sort of meaningful numeric system and analyzing those data....perhaps they'll just sort of get a "feel" for what comments are saying by having employees generally read through them?).
I don’t know about needing to convert anything being very difficult. Most people (and I don’t mean you or anyone specifically) aren’t really as clever and original as they think. So in surveys like this, with a bit for written comments, you’ll get 200 different ways to say the same 5-8 things. Some people will be wordier than others, but the sentiment will be the same. It doesn’t take too long to skim the responses and say 32 people don’t like ardlings flying because flying at level 1 is OP, while another 16 think it’s bad because it’s too weak, and 24 just don’t like Ardlings at all. You basically just make a tic mark every time you come across each idea. It’s not like trying to read Proust and looking for the subtext. It can be time consuming, but it’s not very difficult. It’s pretty much why interns exist.
If anything overall gets ignored, it will be the open-ended one at the end if people are just too vague, (“I don’t like gnomes” but they don’t explain why.) or say things like, please don’t change anything.
To use the written comments as "data" they'd have to somehow convert those to a numerical system that can be analyzed. (Imagine trying to convert the massive volume of written comments to some sort of meaningful numeric system and analyzing those data....perhaps they'll just sort of get a "feel" for what comments are saying by having employees generally read through them?).
I don’t know about needing to convert anything being very difficult. Most people (and I don’t mean you or anyone specifically) aren’t really as clever and original as they think. So in surveys like this, with a bit for written comments, you’ll get 200 different ways to say the same 5-8 things. Some people will be wordier than others, but the sentiment will be the same. It doesn’t take too long to skim the responses and say 32 people don’t like ardlings flying because flying at level 1 is OP, while another 16 think it’s bad because it’s too weak, and 24 just don’t like Ardlings at all. You basically just make a tic mark every time you come across each idea. It’s not like trying to read Proust and looking for the subtext. It can be time consuming, but it’s not very difficult. It’s pretty much why interns exist.
If anything overall gets ignored, it will be the open-ended one at the end if people are just too vague, (“I don’t like gnomes” but they don’t explain why.) or say things like, please don’t change anything.
I'm just not sure how they would handle going through them, if there's high a high participation rate? Seems like there would be alot of participation in this survey. I can't imagine dealing with reading the comments of 100 people and trying to determine what they mean (aside from a simple "good" or "bad", which doesn't capture much context). If there are thousands of comments it would be a huge undertaking. But perhaps there are an army of willing interns pouring through it as we speak.
BUT...I guess that's why the folks in charge of marketing and game design at WOTC "make the big bucks"!!! They get to worry about figuring all of this out! :)
The statistical tools to get insight from free form text these days can be pretty good, in addition to doing an analysis on them. There's many ways to look at the same data and get insight, find things to "zoom in on", sentiment, word clouds, etc. Plus, you eliminate all the folks who don't comment on a page. It'll take less people than one would think. Well that's an approach anyway, there's nothing stopping them from reading every single one and having folks manually classify the data. I guess my point is, they'll read it one way or another. :)
Keep in mind that part of the reason we don't have the entire Codename: One D&D ruleset is that we're also supposed to also test the backwards compatibility of it—use the new stuff as updates to existing stuff in 5e and keep the rest of 5e. Don't wait for the entire ruleset to playtest.
Rollback Post to RevisionRollBack
Human. Male. Possibly. Don't be a divider. My characters' backgrounds are written like instruction manuals rather than stories. My opinion and preferences don't mean you're wrong. I am 99.7603% convinced that the digital dice are messing with me. I roll high when nobody's looking and low when anyone else can see.🎲 “It's a bit early to be thinking about an epitaph. No?” will be my epitaph.
To post a comment, please login or register a new account.
That's true....hadn't thought of that Pocketmouse. I always wondered how well those free text data conversion tools/software work? Can they really get the nuances of context with them? I'm just used to dealing with continuous datasets, I suppose.
They're not going to carefully analyze thousands of comments, but it would be technically trivial to assemble all the comments from a particular page into a single large document and then read through it looking for common themes. That will probably miss comments that aren't made by a lot of people, but, well, comments that aren't repeated probably aren't very important. They could add things like word clouds to do more sophisticated text analysis, but it's not really needed.
Yep. That makes sense too....
Actually, it would be kinda cool if they released a report someday that shows how they analyzed feedback and what the results were. If they've done this before, I've never seen it? It would also only appeal to some folks, I suppose. Still, would be interesting!
As an aside, I figure the more concise your comments are, the more likely someone is to actually read and process them. I can envision someone's eyes glazing over at a comment box filled with tons and tons of text.
I don’t know about needing to convert anything being very difficult. Most people (and I don’t mean you or anyone specifically) aren’t really as clever and original as they think. So in surveys like this, with a bit for written comments, you’ll get 200 different ways to say the same 5-8 things. Some people will be wordier than others, but the sentiment will be the same. It doesn’t take too long to skim the responses and say 32 people don’t like ardlings flying because flying at level 1 is OP, while another 16 think it’s bad because it’s too weak, and 24 just don’t like Ardlings at all. You basically just make a tic mark every time you come across each idea. It’s not like trying to read Proust and looking for the subtext. It can be time consuming, but it’s not very difficult. It’s pretty much why interns exist.
If anything overall gets ignored, it will be the open-ended one at the end if people are just too vague, (“I don’t like gnomes” but they don’t explain why.) or say things like, please don’t change anything.
Another good point! Thanks
The statistical tools to get insight from free form text these days can be pretty good, in addition to doing an analysis on them. There's many ways to look at the same data and get insight, find things to "zoom in on", sentiment, word clouds, etc. Plus, you eliminate all the folks who don't comment on a page. It'll take less people than one would think. Well that's an approach anyway, there's nothing stopping them from reading every single one and having folks manually classify the data. I guess my point is, they'll read it one way or another. :)
Keep in mind that part of the reason we don't have the entire Codename: One D&D ruleset is that we're also supposed to also test the backwards compatibility of it—use the new stuff as updates to existing stuff in 5e and keep the rest of 5e. Don't wait for the entire ruleset to playtest.
Human. Male. Possibly. Don't be a divider.
My characters' backgrounds are written like instruction manuals rather than stories. My opinion and preferences don't mean you're wrong.
I am 99.7603% convinced that the digital dice are messing with me. I roll high when nobody's looking and low when anyone else can see.🎲
“It's a bit early to be thinking about an epitaph. No?” will be my epitaph.