Before anyone gets bent out of shape, let me explain what I mean. Unlike the film industry, we don't have full time (or part time), paid for by a third party (media / news outlet) reviewers / critics. Heck, we have few full time creators in the RPG industry. Most folks that create do so on the side - they have "real jobs" that pay the rent, put food on the table and hopefully offer health benefits.
Our "reviewers" are pretty much the same. They review on their blog, podcast or YouTube channel on their own time. Nobody pays them to review products (that's where affiliate links come in - that's fodder for another post).
So, what does all this have to do with a noticeable lack of negative product reviews?
Simple. There are basically two ways for a potential reviewer to obtain a product for review. The first is to buy it themselves and the second is to be comped / gifted a copy to review, usually from the distributor (OBS / OGS) or from the publisher / creator themselves.
If I'm going to buy something for review purposes, I'm going to pick something that I have high expectations of. The publisher / creator will have a history of releasing top notch work. It's likely to be something I could see myself using in my own campaign.
What I've done here is I've filtered the results of the potential review pool. I am, most likely, going to have a product that will have a positive review.
What about review copies? If you are a reviewer for OBS, you likely get numerous emails each day containing links to comp copies of products to review:
Now, how many of those will I actually download and review? Very few. One in a hundred, maybe. It really has to strike my interest to want to grab a copy and review it, even if it's free.
Why?
Because its no fun reading crap and generally speaking, its even less fun writing a review about crap. So I self filter, again. Looking mostly for OSR releases and even then, looking for something that I may actually use at my table.
Products aren't chosen for review by a random lot, at least not by me. They are chosen based upon my interests.
If they were random, I'm sure there would be a spike in negative reviews. Come to think of it, if they were random I'd be less likely to do them at all. As stated earlier, writing reviews of RPG products generally isn't a paying gig. Affiliate links to the products reviewed aren't just about the immediate sale but the long tail and later sales. I don't post reviews hoping to sell copies at an online store. I do hope that folks will be interested enough to check out the online store and maybe make a purchase down the line that suits their interests.
I do post some negative reviews, but that isn't by design. I just happen to grab the occasional product that, despite my best hopes, isn't up to par.
As stated, reviewers generally are filtering what they will review, not how they will review it. That initial filter does keep the number of negative reviews in check.
Interesting...from my perspective though I would contend that outside negative reviews I wish we had a system of, say 1-5 stars, that would help differentiate the good stuff (1-2), the above average stuff (3-4), and something that truly differentiates itself as a uniquely superb product (5). Or something like that. Otherwise it seems like some reviewers give almost everything a 5 - that is not useful to me at all.
ReplyDeletewhich is why I don't give stars, mugs or whatever.
Deletenumbered ratings are only useful there are a large number of reviewers of said product, like on amazon and to a lesser sense, OBS
Star ratings aren't particularly useful because they don't tell you why the reviewer gave the product that rating. It's better to just put a tl;dr summary of your review wherever you would put the number of stars.
DeleteThe exception that proves the rule: http://tenfootpole.org/
ReplyDeletethere is always an exception ;)
Deleteof course, Bryce reviews much more than the average blogger
Realize the other part as well. If you, as a reviewer, appear to post an inordinately large number of negative reviews, the chances of you being sent more product by the same manufacturers is low. And others will likely steer clear of you as well so as to avoid the chance of another bad review. So reviewers often go by the "if you can't say anything nice, don't say anything at all" unspoken rule. I personally have only written negative reviews when it wasn't obvious from the package and I felt the need to warn others...:D
ReplyDeleteI included the snapshot of my email account to show that, literally, i have no lack of potential review material, just a lack of material I desire to review.
DeleteI have no problem saying something not nice. I don't fear being cut off from review material. I just don't go out of my way to find lumps of coal. I look for gems. Some are flawed and some are worthless, but I always want to find a valuable gem.
I love Bryce Lynch's reviews at tenfootpole.org. Any time I see a new post from him I read it.
ReplyDeleteIt is my sincerest wish that you will continue to find gems whilst sifting through all of the coal that's out there. You do a mighty fine job of it, Erik. Thank you for doing so.
ReplyDeleteThank you for doing what you do, this is IMO a great essay and I think looking for gems is the way to go. Publish good things and ignore bad things (except when you get fooled, then call it out if warranted).
ReplyDeleteI think affiliate links do muddy the water a bit, but not much. I think you can get a pretty good idea of what is real praise and what isn't, just by reading the piece that the reviewer publishes. I have done one actual review and it is one of my most viewed posts, so I do plan on doing more. If at any time I am privileged to have any affiliate links I will definitely post notice that they are there. That is just Basic Honesty 101. Any reviews I publish will be stuff I bought or stuff that is free to everyone, I would be very surprised to have something sent to me for free to review, but if I did I would disclose that at the beginning of the review.
When I started doing reviews for another website (honestly don't even remember the name) there was a dibs protocol for selecting product. When I noticed that what I was doing was just driving traffic and affiliate links for someone else I decided I'd just do them on my own blog instead.
ReplyDeleteI did end up posting to OBS my general reviews, but I never reviewed material that I'd say was competing with materials I was producing. There were some bad products (I recall one that outright lied as to what was being offered and another that padded the download so badly only 25% was what you bought) and I didn't hesitate to leave appropriate reviews on OBS.
OBS quickly told me I cannot leave other publishers reviews....period. They didn't seem to have any problems with the good reviews, only the bad, so now I only review on my blog.
I do freely admit, and have multiple times online, that my reviews are naturally skewed positive because I usually review items that interest me. If I have to pay for stuff I'm not going to throw money at something I'm not interested in. I have been given product to review, which I always disclose, and I'd like to think I've been fair in the reviews I give.
Of course everyone thinks their opinion is "fair". I know I have certain biases and do what I can to acknowledge them. If anything, I like reviewing because it often helps me look at what I'm doing (creator wise) a bit more closely.
I should add though that I don't disclose affiliate links on every post. I do have a blanket statement on my links page though. I do go through and occasionally look at the affiliate links commission and what little I get comes from ancillary purchases, not (really) on the items I've reviewed.
DeleteMy last 'mixed' review was something like: 'Death and Taxes: the most beautiful and well produced module that I could possibly never have the time, energy, patience or smarts to even try running for my group.'
ReplyDeleteBut yeah, like Christopher says above me, OBS will chastise you if you leave negative reviews and are also a publisher (even if you have a publisher account but haven't actually published anything!)
Well, I can understand the problem with publishers leaving bad reviews (and yes, it should be ALL reviews; but we all know the TripAdvisor story, right?)
ReplyDeleteAlso - as I used to be associated with other fields too - the "not many bad review" things happens for reviewers with a salary. If you work, say, for a music magazine that receives 200 records/week for review, and you are known to love country, most of the country records will fall on your table; and among them, you'll go first for the artists you like, then for those you don't know but you want to know and THEN - if there's time - for the rest.
Ditto for novels.
So, in whatever field you are, unless your boss has a strict policy and assigns you reviews out of your comfort zone on purpose, most reviews tend to be positive (and the negative ones tend to be devastating, because you WANTED to like that, but you didn't...)
I just have no desire to do negative reviews. I'd rather single out a product I enjoy than shit on a project someone presumably put time, effort, and love into. There's enough negativity on the internet already.
ReplyDeleteYou are missing the point on how to do proper reviews. You take Product X, read it, re-read it, play test if possible, FIND FLAWS, IDENTIFY MISTAKES, critique maps, the handouts, the layout, etc...
DeleteThe point of doing a review is not to cherry pick from your favorite sources of product because transparency then is already impacted with your bias.
If you have the time for such an in-depth review process, I'll offer my soapbox for you to post them on...
DeleteYep...I have so little time for such that I need to focus my effort on what I actually like. If something doesn't "do it" for me on initial inspection, it's done and over with and I am not qualified to say much more about that product than "I didn't have time to figure out if it was really good." And that's not helpful....so instead I will take the time to talk about the stuff I like --a lot-- because that's the stuff worth my precious free time.
DeleteI think a good reviewer tells it like it is. Bad or good. While I think he has a tendency to be an asshat RPG Pundit seems to be a straight shooter. While I don't always agree with his assessments, he's not a yes man by any means. I can totally respect that.
ReplyDeleteI've written a couple negative reviews. I look at it as a warning to potential buyers. I've had a couple things that "reviewers" say is great, but the reality is considerably different.
ReplyDeleteIn one case (an item by Skirmisher Publishing), the item was one of the few things I've deleted from my computer as a waste of space. The publisher wrote back a scathing response, accusing me of all sorts of things, including living in my mom's basement. To their credit, OBS deleted his response and awarded me a gift certificate.
I'll also note that I usually find the negative Amazon reviews more helpful than the 5-star positive reviews.
I don't review items often because I've limited my selections to a small number of publishers who consistently put out good stuff, and another positive review on top of someone like Thilo Graf (Endzeitgeist) just seems like overkill.
There's one or two reliable semi-pro reviewers in the science fiction community supported by patreon; as long as they're receiving a regular stream of support, enough to cover time and purchase, they post a review (positive or negative) every day or so, and as a result manage to cover a fair bit of stuff. I suppose that might be a doable model for someone with a decent reputation as a blogger.
ReplyDeleteUnbiased regular reviews sort of died out with the demise of the independent print game magazine - the magazine may have got free stuff from the companies, but they just sent 'em out to freelance reviewers and there was no pressure to write positive or negative reviews. I remember when Space Gamer, back in the 80s, used to try to review just about everything (albeit for a much smaller field) and did a decent job with 20-30 reviews/month.
Several other magazines made a go of it into the 90s. The main problem (even before the demise of the print magazine) was that the good magazines tended to turn into game companies and have no time for reviews - that happened to White Dwarf (->Games Workshop), Space Gamer (->SJ Games), and White Wolf magazine (->White Wolf).
Although I was originally annoyed by Bryce's negative review of one of our products, I read his review standards and we made changes to our product line to reflect those standards where Bryce and I agreed.
ReplyDeleteWe're not interested in just getting positive reviews, that doesn't help anyone, not our customers and not us. I wish there were more people doing reviews.