If you’ve been following UploadVR for a while now (first of all: thank you!), you might be familiar with the rollercoaster ride that is our reviews system.
In the past three years, we’ve ditched 10-point reviews, moved to a five-star rating, then abandoned the stars and relied exclusively on a five-tier labeling system. Why so many changes? And are we done yet?
With our latest (and hopefully final) review system, we’ve opted to do away with numbers entirely. In our experience, numbers just don’t do VR justice. The bar for quality is constantly shifting, as is the volume of content coming our way, and some of our older review scores are no longer fitting representations of what we feel about a certain product. We removed our star rankings in 2020, but we’ve seen enough feedback now to know that, just because we say a game is ‘Good’, doesn’t mean our readers believe us (I promise you I do actually like Lone Echo 2).
It didn’t help that Metacritic took our scores at numeric value, giving anything below our ‘Great’ and ‘Essential’ rankings the dreaded yellow or red marks. With this new system, we’re looking to see our assessments placed in the unscored review section so they no longer contribute to a flawed system that may encourage publishers to assign bonuses to developers based on aggregate scores.
So we’ll now convey our recommendations and criticism in four areas: software we think is ‘Essential’, those apps we ‘Recommend’, and some we think you should ‘Avoid’. In addition to these, will also have a large number of reviews without specific labels.
You can read more on how we decide on those labels in our updated review guidelines, but this final category will consist of experiences that might have lots of great mechanics, redeeming features, and/or compelling ideas, but are weighed down by some road bumps that prevent us from easily recommending it to most people that own a VR headset.
It’s not to say that they’ll be bad for everyone that tries them, but withholding a label and a number from our criticism of these works hopefully reflects that we expect the reception to this software to be generally mixed among VR headset owners with widely varying tastes and interests.
This system might look a little familiar to you, and that’s because we’re following a template laid out by the Eurogamer team and their industry-leading criticism. It’s a smart system, and we aim to model our criticism in a similar fashion because we think it’ll give us the confidence to share our perspective more clearly and directly while still being fair.
After all, the reality of VR design is that a piece of software at release may be very different from the same title a year later, and while a project’s direction and a developer’s plan don’t change the reality of what a person experiences day one of release, those intentions and plans are still relevant to a buying decision. We think this change will give us the confidence to assess all these elements and communicate them with clarity and fairness.
It’s been a persistent battle for us to develop a system that does this. We need to assess the broad range of media types emerging in VR (not just gaming but also movies, fitness, productivity, and more), and stack that against an evaluation of the hard work that (most) developers put into their craft while still providing a fast and efficient means of communicating to our audience if a product is worth their time or hard-earned cash. This system, we believe, may be as close to a one-size fits as we can get.
So this is us trying something new… again. As with past systems, we’re testing the waters and seeing where it gets us, but we’re hopeful that this will be the final form for UploadVR reviews that carries us into the future.
