Been doing a lot of thinking recently on the current homebrew rating system. I'm was curious what you guys thought about it and if anyone else has come up with their own any ideas on how to improve upon it or if they think it's fine as is. One thing I have noticed after some time is that homebrews usually take a while to get rated naturally (without making a posts either asking for opinions or the such), but once a homebrew gets a rating or added by other people, it seems that people are to be more willing to rate on it then. With the current system, it is simple which comes with it's own benefits and drawbacks. Such as because of this system is simple more people should be more willing to rate content because it's easier, but then the creator doesn't know why their homebrews are getting the ratings that they are.

I think I would prefer a 1-5 rating system with three parts focusing on: Originality/Creativity, Clarity/Grammar, and Balance. Though I'm honestly not entirely sold on the idea of having balance as one of the focus points, simply because everyone has their own opinion on what a "balanced" homebrew is.

Commenting on the homebrew would be more useful to be honest. Not sure an analogous ThumbsUp/ThumbsDown would mean anything. Like...what does "Balanced" mean to one high magic setting versus a low magic one? Admittedly, 5e would be easier to "balance" if it was more linear, or at least the damage curves could be predicted more easily. This isn't even getting into "balancing" social/exploration homebrew.

At best grammar and spelling isn't as subjective.

I despise the rating system. It's inherently arbitrary and doesn't promote helpful feedback. Most will look at an entry and rate without leaving any feedback, which results in negatively-rated items being pushed farther back so that they never get attention they may deserve.

I think the issue is that the goals of a rating system were never clear from the get-go. Are we supposed to rate things highly for balance and design? For the "cool" factor? Or was it meant to be used as a bludgeon against the mountains of garbage that populate most of the homebrew listings? In the former case, they don't do a good job. In the latter, it's woefully inadequate.

If it were up to me I'd remove ratings altogether, keep the adds metric, and have the mods more zealously take down incomplete homebrews. If people really need a popularity contest, I think a curated section or weekly spotlight (which was floated by a mod in the distant past) would be better.

The vast majority of people do not rate things from 1-5, it’s usually just all 1s and/or 5s. That’s why even Netflix did away with their 5-Star rating and moved to a simple up/down vote. People actually vote more often when it’s as simple as up/down. If they moved to a system as complex as you described, more people would stop rating things altogether.

I don’t think that we can rate homebrew effectively or efficiently. There’s too much of it and more comes every day.  Everyone looks for different things in homebrew. Some people look for goofy one shot ideas, others look for character concepts that aren’t in the game yet. A weekly spotlight might work but it would take more than one mod doing it once per week.

There's a truism that explains this well, methinks.  It comes from fantasy sports, but it applies broadly to many different creative endeavors.

"Nobody is more interested in your [character/homebrew/fantasy Sportsball team/whatever else] than you.  Nobody is less interested in somebody else's [thing] than you."

People are invested in their own creations in a way they generally will not be in somebody else's.  Even close friends often have to badger each other into being truly interested in each other's creative endeavors, and usually requires some buy-in on the creative process.  People are interested in the other characters in their games because those games are a shared creative project as well as a fun game, and so a good player naturally wants to learn more. 

But for unrelated homebrew stuff, especially on DDB here where the floodgates are open and the nuggets of actually useful design are drowned in a sea of poorly written, worse designed, broken and nonfunctional (in the sense that it doesn't work for spit in the sheet, as well as the fact that it's not even pretending to be properly balanced out) utter bottom-shelf trash?  I'm not surprised nobody gets any of the written feedback they're looking for without specifically demanding it.  Hell, even here in this forum, more often than not somebody's homebrew idea goes completely un-commented on unless they're doing an Animeme Thing.  Unless it's somehow relevant to their own games and their own creativity, people just don't care enough to blow their time on other people's homebrew when they could be working on their own homebrew, instead.  Or something else related to the game.

No rating system will fix that.  No amount of oversight or cleaning-up-of-garbage will address that fact.  Some folks like to play Homebrew Curator, and I honestly really like the idea of a Weekly Spotlight or some other means of occasionally shining a light on really cool homebrew ideas that aren't Two Goddamned Goblins in a Goddamn Trenchcoat.  But the fact that "Two Goblins in a Trenchcoat" not only exists but is one of the highest-rated pieces of homebrew on the entire site just goes to show that the rating system is mostly unrecoverable.  Would be nice if it worked the way it was intended to, but it never will, because people just don't have the time or inclination to care about other folks' creativity.

@FallenPhoenix     I appreciate the general idea.  Having specific metrics by which to think about homebrews can be helpful.  At the same time, I am inclined to agree with much of other people's opinions in this case, as there is both too much homebrew content on DDB and too little incentive to take most of it seriously.  It would work better in RPG circles where everyone has gamed with each other and therefore are more inclined to have some social-emotional investment in each others' content.  So the most likely "lift off" method of starting up something like this would be to use the peer-to-peer networks that one already has in the gaming community.  It's not particularly equitable, though, b/c some people play at far more tables and have a wider pool of experienced gamers from which to draw from, but the reality is that most social media, including DDB, isn't very conducive to creating the social bonds that would facilitate the frequency and depth of creative sharing necessary for it to work for any major length of time.  

I think that a simple like/dislike system is fine, but as a creator I would prefer for it to be tied to comments as it's disappointing to get a negative rating and have no idea why.

When a user dislikes something I'd like to see them forced to write a comment at the same time when doing so, so that I (or anyone else) has some idea of what could be improved or clarified.

To give a specific example, I have a feat that has been given a dislike but I have no idea why: Gentle Giant (Large)
But I don't know if that person objects to the third bullet point (spend rounds taking no hostile actions in combat to get a bonus to attacks later on), which at first glance sounds powerful but actually isn't, as most combats don't last long enough to max out the bonus, while others each round you "skip" is a round where you are doing no damage. So did they think it was OP? Did they feel the feat wouldn't/shouldn't have a combat related benefit at all? I just don't know, and that's quite frustrating, as I'm quite happy with how it works in terms of what I wanted to achieve with it.

Possibly to do with the “large” prerequisite since there are no official large PC races. 🤷‍♂️ It wasn’t me, so I cannot say for sure, but that’s my guess.

Yeah, the more I'm reading all of your responses and I'm more inclined to agree with the single +/- is probably the best voting method for now. Even though I know that plenty of us would like more insight on why people like/dislike our stuff but just what Song_of_Blues was saying, DDB doesn't seem to promote this behavior either. Like for most of the input that I get for my homebrews is from what I post either in this forum or the class forums.

It has been brought it a few times here, but I would like to see the homebrew spotlight to be brought back because not only does it help with some creator's getting their stuff noticed, but it also shows other creators stuff that they might not have made themselves.

As for your feat @Haravikk, the only thing that I can think of is like what Sposta was saying... you looked into adding other larger races like goliaths, and firbolgs?

I've even been a victim of someone going through all of my homebrew content and down voting everything I had made at the time. All of my current down votes are still remnants from when that individual did that.

Quote from FallenPhoenixIV>>

As for your feat @Haravikk, the only thing that I can think of is like what Sposta was saying... you looked into adding other larger races like goliaths, and firbolgs?

Yeah, I actually have some Large races (Minotaur and Troll currently), but since you and IamSposta are thinking the same thing I've added a comment to the feat in case that helps. I would have provided some kind of link within the feat but "external links" prevent sharing, even if the link is to D&D Beyond! *facepalms*

I've even been a victim of someone going through all of my homebrew content and down voting everything I had made at the time. All of my current down votes are still remnants from when that individual did that.

This is actually partly why I think it might make sense to require a comment on a downvote, as if someone was just being abusive/trolling or whatever then a comment might make that more obvious, in which case it can be reported (and ideally if the comment is removed, so too would the downvote).

Quote from FallenPhoenixIV >>

I've even been a victim of someone going through all of my homebrew content and down voting everything I had made at the time. All of my current down votes are still remnants from when that individual did that.

 Same. I thought my Magic Apprentice feat(s) were pretty good. 🤷‍♂️

I think vote+comment is good, but I think there should be an easier way to flag incomplete homebrews for removal. I recommended that the system auto-reject any brew that includes placeholder text (see this for an example).

An interesting thing I've discovered, posting occasional homebrew here in the forums, is that many things I've thought were surefire locks were nothing but.  A variation I worked up for Counterspell that I was rather hubristically proud of and thought was a splendiferous fix to the spell was shot down pretty firmly, and once I got over my wounded pride, it was actually quite helpful to see.

That's the sort of feedback people want.  And if you want it, the only heckin' way you're going to get it is by prodding people in the forums, here.  Any comments you get on your homebrew just randomly out of the blue should be treated as the lottery winnings they are.  Even ratings are honestly really rare, it seems.  If not for the fact that the few things I've published have been added a few times I wouldn't even know if anyone had seen it.  A simple up/down rating system is the best way to get ratings, and comments?

You may as well forget they exist.  Sadly.

This was my best idea to solve the problem:

https://www.dndbeyond.com/forums/d-d-beyond-general/d-d-beyond-feedback/30499-request-and-vote-for-new-features-for-ddb?comment=951

If you like it please go upvote it.

Quote from IamSposta >>

This was my best idea to solve the problem:

https://www.dndbeyond.com/forums/d-d-beyond-general/d-d-beyond-feedback/30499-request-and-vote-for-new-features-for-ddb?comment=951

If you like it please go upvote it.

I do like that idea of a soft publish so we can share for review purposes and possibly allow people who are looking it over to see how they set up their modifiers. It would be so much easier for something like this to exist so you can send a link instead of either sending a wall of text or a series of images for someone to look over.

Which also raises a different issue, why aren't we allowed to make our content no longer publicly visible? I understand that the people who already added the content shouldn't lose access to the content but why not just have it hidden like how sub races currently are.

Oh, one other thing I thought about for the rating system, but in addition to requiring a comment, I wonder if it makes sense to remove the vote up/down buttons from the search results page? i.e- to rate a homebrew you must go to the view details page, where you can see any existing comments, notes by the author etc.

Might not seem a big change, but that extra click could make the difference between an impulse vote and a considered one?

The issue is that some people are desperate for ANY votes, impulse or otherwise.  It's already like pulling teeth from an Ancient Red Dragon to get any feedback whatsoever on most homebrew one makes around here.  You have to actively beg for it.  Any barriers between Random Doof Homebrew User and Feedback will slim that down even more.  Obviously the ideal is considered feedback from direct play experience with a comment, but man.  I'd really like to find a five-pound chunk of gold while walking down the street one day too, and both of those seem equally likely.

Quote from Yurei1453 >>

The issue is that some people are desperate for ANY votes, impulse or otherwise.

I feel that pain, even with only a small handful of published homebrew items!

But I'm kind of of two-minds; I think ill-considered up-votes are no more useful than ill-considered down-votes, especially when the number doesn't necessarily tell you anything. I know I'll never be able to find it again but I remember reading a while back a study about rating systems, which I think was about whether Facebook, Twitter etc. should have "dislike" options, and it went into issues like for example on Twitter where something that gets liked a lot gets into a kind of "likes feedback loop" where because it's popular, people are more likely to like it without properly considering if they actually do like it, kind of like wanting to fit in psychologically/giving in to peer pressure.

Another interesting possibility would be to split apart the up and down votes into "reactions". For example, you might have "cool", "balanced" and "fun" as positive reactions (upvotes) and "needs work", "unbalanced" and "breaks immersion" as negative reactions (downvotes). The idea is that this way you're emphasising why you like/dislike something, and on the details page there could then be a breakdown of the reactions to show how many voted an item as balanced, needs work etc., so it would give some idea what the feedback actually is. These are just some example options, definitely would need to be some thought about which options would give the most useful information to creators, and "breaks immersion" is a bad term but I can't think how else to describe something that doesn't fit like 4th wall breaking shenanigans. Alternatively there could be emojis if there are good choices to convey a particular response (like thumbs up/thumbs down for basic like/dislike, but also sunglasses for cool, puzzled for strange/out of place idea and so-on).

Still though, I think that comments should be encouraged if not required, as comments and forum threads are the best feedback when you can get people to engage with them. Maybe if instead of instant-rate, if there was just a "rate" button that opened up a rating window with the complete options and a comment box, all of which have to be filled in to submit your feedback?

Quote from Haravikk >>

Another interesting possibility would be to split apart the up and down votes into "reactions". For example, you might have "cool", "balanced" and "fun" as positive reactions (upvotes) and "needs work", "unbalanced" and "breaks immersion" as negative reactions (downvotes). The idea is that this way you're emphasising why you like/dislike something, and on the details page there could then be a breakdown of the reactions to show how many voted an item as balanced, needs work etc., so it would give some idea what the feedback actually is. These are just some example options, definitely would need to be some thought about which options would give the most useful information to creators, and "breaks immersion" is a bad term but I can't think how else to describe something that doesn't fit like 4th wall breaking shenanigans.

 I like the idea of breaking up upvoting and downvoting into “community tags”. It would also make sense for authors to self label to a point that way when you publish, you can highlight your homebrew as “goofy” or as an “archetype”.

The discussion of what those tags would be, what they mean, and how to best implement them would likely spiral into a long recursive thread. And if those changes were made, we would likely see after a year or two of that system, a very similar thread to this one declaring that the system has failed.

Side note: at the time of this posting there are nearly 270,600 homebrews available for perusal on DDB. If you spent 5 minutes reading, evaluating, and commenting on each homebrew, it would take you 2 and a half years without stopping.