Valve says it fixed “collection and analysis” issues with the Steam Hardware Survey’s VR section.
Last week we published an article outlining recent anomalies in the Steam Hardware Survey’s VR section, including large unexplained jumps in the key ‘Steam users with VR Headsets’ figure. We concluded that we’d no longer be using it as a reliable source for our analysis of the VR market.
We had reached out to Valve for an explanation on multiple occasions – hoping for a fix – to no avail. But now Valve has responded, issuing corrected data for the past three months and saying it has fixed “collection and analysis” issues affecting the accuracy & consistency:
“The hardware survey sees natural variations in survey response rates from month to month, so some variability in the statistics is to be expected. However, the May/July jumps and extra category in August had us take a deeper look, and we’ve now fixed a few issues with the collection and analysis of data, as well as corrected the August survey results page. We expect to have more accurate results going forward.”
The Hardware Survey is offered to a random sample of Steam’s userbase each month. If you choose to accept, it uploads your PC specifications and peripherals. Before March 2020 the survey relied on headsets being connected via USB at the time of sampling, but Valve changed it to scan your SteamVR logs from the past month. Companies like Meta, Valve, & HTC don’t reveal hardware sales figures so it has been the most reliable indicator of PC VR’s adoption.
Here’s the ‘Steam users with VR Headsets’ data before correction, as seen in our previous article:
Now here’s the corrected data we received for May, June, July, and August:
If Valve’s claim of more accurate results going forward holds true, we’ll resume using it for analysis of the VR market. We’ll watch the data closely in the coming months to verify this is the case.
