Where do VG Chartz North American Numbers Come From? - News
by VGChartz Staff , posted on 10 November 2007 / 13,434 Views
Some common misconceptions I want to put to rest.
1. VG Chartz gets no data from retailers.
Wrong. How does anybody think we would ever be approaching the level of accuracy and putting out the quantity of data we do each week if we didn't get data from retailers. The "leap of faith" many struggle with is why would retailers provide us with data? Which retailers? How much data? Why can't we say? What are we hiding?
Well unfortunately if we were to release detailed information at this stage on exactly where we get our data from then everyone would jump on the bandwagon and competitors would likely get their cheque books out and try to block anyone from providing us with any more data. Suffice to say (and this is where I do have to simply ask people for trust) that we do get data from a number of small retailers. Anyone with a background in statistical analysis will know that you don't need a large sample size to get good results, merely a good sample. For the small amount of data we collect, we get good results because we get a good mixture of data from different retailers - each of which represent various portions of the market - typically split into specialist gaming stores, electronics stores, general retailers, toy stores and online retailers. We have representitive data from all of those "areas" of the market and scale data up to represent the marketshare that each "area" represents of the total. This happens across both America and Canada.
Why would retailers provide us with data? Data exchange, many retailers are keen to know where they fit into the market - this is NPDs business model, they don't "pay" retailers for data they exchange data from one retailer for breakdowns of the entire market. This is far more valuable to a retailer than any monetary payoff and is exactly the reason why Walmart don't work with any independent tracking services (or do they?). This is also the same method that Neilson are hoping to use if they ever enter the market.
We can't go after the larger nationwide retailers (as yet), but we certainly have some interesting partnerships with smaller regional and independent stores, startups and so on. I hope people understand why at this stage they have to remain under wraps - expect official announcements in coming months though. We are also implementing premium advertising schemes for new partners - a community of over 10,000 avid games and 4 million impressions per month is a big draw for small retailers. Especially since you can now target advertising to gamers based upon games they already own...
This is where site expansion (in terms of number of visitors) will only make us stronger.
2. VG Chartz data is "based on" NPD data - they are stealing and benefiting from the work of others.
VG Chartz releases weekly sales figures on the Tuesday following the Saturday of the close of the previous week. In 3 days we collect an enormous amount of data from different sources, analyse it, extrapolte it, make comparisons and checks and put the data up online. This is a huge effort from a very small team of amateur (in that they have other jobs) but very able (in that they are highly educated and experienced) people who are doing it for very little payoff. We have been doing this for 18 months now, at the same time as going back and adding older data - some of our partners have been supplying us with their historical data as well, and have been refining our methods week on week.
Do we adjust our data? Not as such. Do we adjust our methods then? Yes - which will of course alter some data. On what basis? If we believe that a particular data set differs significantly from other sources of data (data released into the public domain by tracking firms, manufacturers, analysts) then we do re-check our data and make adjustments to the methods / scaling factors used. This happens on a fairly infrequent basis - less often than we adjust due to internal data changes - and is something that every tracking firm and analyst does. I personally have no issues with "benchmarking" our data from time to time against other sources of data - as long as it has been made public. Note, however, that any decisions by NPD to stop releasing public information will not affect our figures or service at all in my opinion - I am more confident every week in our own data and relationships with publishers that our data will only go from strength to strength. It is certainly no loss to vgchartz!
3. VG Chartz data is inaccurate and worthless.
Well this obviously depends on personal opinion. But for me, a site that gives sales figures to such a depth and accuracy as we do here, at no cost to any user is anything but worthless - especially with all the powerful analysis and charting tools available.
If we get figures to within, say, 15% accuracy in a given week (understand that statistically weekly inaccuracies are likely to cancel out over time, not accumulate) then I am happy. We pegged Mario Galaxy in Japan based of one retailer at 285k, it came in at around 255k according to Famitsu / MC - that's close enough for me. It tells me how well the title performed. It didn't do 900,000 like NSMB, it didn't do 100,000 and bomb completely. It sold a similar amount to Mario Sunshine and was quite disappointing - although should hold well over the holidays. I can make all these conclusions whether it sold 255 or 285k. Same as when we put the latest Tony Hawk figures out. Very poor first week. How do we know that if we're not getting "real" data? If we'd have predicted based on last years trend we'd have it a lot higher. I'm sure we'll be 10-15% different to NPD but so what? You know the game performed poorly. We had Bioshock sales way above expectation - good guess was it? We had Metroid Prime below Bioshock - again spot on. Too much is made of 10-15% being inaccurate - it is enough to give a good idea of how a game performs, and that is what we are all about.
If we put Mass Effect figures at 75,000 when it comes out people will be in uproar. They expect 300k+. If figures are confirmed a few days later at ~100,000 are people going to accuse us of being "inaccurate" because we were 33% off? Surely to have even put our figure at less than 100k is an achievement and shows that we have far more insight into sales than anybody else. Expecially when NPD might only, for example, have it at 85,000 themselves. I think a bit of perspective is required here on what is accurate and what is not - especially where software is concerned. To consistently get within 10-20% is damn good. Again, I ask for trust here, but from spending over 10 years looking at videogame sales data then to be getting this kind of accuracy week on week is far beyond where the professional tracking firms themselves were only 10 years ago (you'd be lucky to get 30-40% accuracy).
I hope this clears a few things up. Anybody with half a brain can see how exciting what we are doing here is and the enormous potential.
Any questions then please ask. If anybody is in a position to potentially help us out further then please get in contact.
More Articles
Your guys are doing a great job, there will always people that don't agree with you. "Success breeds enemies" This is so right. Just keep it up
The numbers here are more than good enough for me. Its not about perfection - its all about decent estimates. Combine that with excellent charting and viewing tools, and you have a winner.
As the saying goes - "If you don't like what you see - change the channel". No one is forcing people to come here for data.
VGChartz is the best thing on the internet. I even lurk here in my sleep.
I have only onme question:
ioi, have you ever been to IOI?
who cares where its from says who cares where its from :)
i have been browsing this sight with my buddies for awhile now because we are working on a very similar type of site to launch in jan. i will send links soon for u to check. anyways we always thought u kept clear of europe charting because it is known that gfk does not provide its sales information in europe the way they do is australea and the others do everywhere else. no stats is europe means no way to generate estimates for vgchartz.com in europe. no offense is intentioned but we use the site for so much more. just keep to what u are doing. these other businesses u feel are after u are mad because they think u take what belongs to them? their data does not belong to them or anyone. it is for the video game busineses and there is nothing ilegal with using their numbers. keep on your toes and 1 step ahead of them.
I believe the NPD group is mad because they pay their statisticians alot of money to do what you are doing for basically free.
They obviously have a larger sample size, but if Political polls only need 1,000 people for like 150 million registered voters, obviously sample size doesn't matter after so many samples.
I think most of the people complaining either havent taken any Quant Methods (Statistics) classes in high school or college, or they ARE the NPD statisticians who are worried about losing their job, lol.
this site is good and it will only become better (especially when the big retailers be a part of the sample in the statistics)
It's a shame that you have to keep defending the site when it's a free resource and remarkably accurate for what it is.
A nice explanation, ioi. You are preaching to the quire but at least it provides a link. The thing is most naysayers don't debate, they just make broad sweeping generalizations without any kind of reason, or vague reasons at best. Much like FilaBrasileiro and diabloz. No reason, just "these numbers are made up! I don't know why I say that, and even contrary to the evidence that they are very accurate on a week to week basis, I still say you just make it up!" or "I have a source that says a different number! I like that number better so yours are wrong!".
And unfortunately you can't really convert people like that. And even more unfortunate, if you pop into gamefaqs or neogaf, you see the internet it made up of morons that have no logical faculties, so much as blind fanboyism, and unfounded opinions that they believe to be facts.
But That shouldn't stop you from trying and spreading the word. This site is getting alot of recognition and it deserves every ounce of it. keep up the good work guys.
Targeting based on games they already own? Finally a reason that might get me to bother with the "I own this game" thing.
There was a saying I learned in graduate school:
"It is better to be approximately right than exactly wrong."
This is important because everyone must remember what VGChartz is trying to do -- to get it approximately right. No one really knows the exact number of sales. Not ioi. Not NPD. Not any retailer or software maker or hardware maker. No one.
However, the basic trends here are evident from the work done on this site. Whether an actual number is 75K or 85K is irrelvant -- especially if the software was expected to sell either 25K or 250K. lThe numbers may not be exact -- but there is no way they can be given and NO ONE should expect them to be. But they do the job and help people (including many people in various parts of this industry) keep abreast of what is going on.
Those who have problems with ithat are typically either ignorant of statistical methods, fanboys of a particular system, or psychopaths.
Mike from Morgantown.
Finally, it appears the only people who don't trust the nu
Success breeds enemies, don't think too much of the people that doubt you ioi. No need to lose sleep over people who seem to get off on bashing VGchartz. Even if you think the site is complete crap...some people go WAY overboard with the bashing, doing it for years at this point...those people should just...not come to the site.
Hopefully accuracy will improve as you get more and more retail information, it's only natural that it will happen as the site gets bigger.
Love the site and everything it brings to the gaming public. Thanks.
The thing I don't understand about people who complain about VGCharts is who do they think does have the proper numbers?
NPD, while a corporate group with significantly more resourses, still produce an estimated number. Do they publish their methods (Do subscribers get this information if it isn't entirely public?) If not, then how is it possible to conclude that the NPD numbers are significantly more accurate?
Do these people believe the manufacturers? Surely the manufacturer has no hidden adgender and wouldn't produce a bigger number for any reason...would they? Even manufacturers can still only 'accurately' produce a number of units shipped. How many have actually been sold is still an estimate to them.
Like you mention, and I feel this need to have more a of focus. The numbers on the site aren't the be all and end all of the sales. They are an estimate and while they may not be 100% accurate, they at least show relative sales, trends and many other significantly more important factors than raw sales.
Does it really make a difference is the PS3 has sold 5.5 or 6 million units? It is still significantly less than the 360 and Wii, and it clearly reflects this. Did it really matter if the 360 and Wii numbers are out slightly and the week VGCharts reported the crossover was inaccurate by a few weeks? No, the data has clearly showed that this was going to occur and shows us that the Wii so far has been a runaway sucess.
People really need to put this into perspective when it comes to the numbers here. If you disagree with them, that is fine, but there is plenty of other benificial information to be gained from this site, and at the cost you really can't complain.
@masschamber: It's not just preaching to the choir, we can use this page as a defense against some criticism which pops up every time vgchartz is mentioned in other sites.
This is a great website and I have learned MUCH about market trends in videogames.
From my experience in retail as a whole, the way you guys are doing it is spot on. And your explanation honestly went above and beyond. You guys have a great site here. Keep up the scrumtrulescent (thank you will ferrell) work.
This should clear "some" things up for the haters (i.e. the people @ Kotaku)... But yeah, there will always be people like FilaBrasileiro and Diabloz. =/
too bad your preaching to the choir, but still a great explanation
@ Diabloz and FilaBrasileiro:
You have any evidence to support your claims that VGchartz is wrong? You guys have better data? If you don't then don't talk
I think we need a site that gives real real numbers because the ps3 has already passed 6 million a long time ago.
Heh, I don't trust this site and I'm a strong believer that the numbers are made up and inaccurate. It's still a good site to get info on what's happening in the VG world though.
Yes, it is true. When ever i post data from this site, on some forum. People go "O that site is garbage"; "Vgchartz? LOL you are joking wright" and etc.
But i trust this site....not all numbers are 100% but it gives you an idea about sales.
Great site ioi keep it up. I've never doubted VGchartz since it began! Keep it up
A big 'Thank you' to you and everyone else who puts this all together. It doesn't get said enough and too many people are quick to judge what you've done. With NPD seemingly 'exiting' the market for free, reliable sales data, I hope that VGChartz becomes the standard that everyone looks to and reports on.
ioi, if you need windows programms to analyze your data, help fine tuning it, etc. just write me. Thats nearly the only thing I could do to help you. (Analyzing data and code programs is my job, after all)
I'm actually more interested in where your Europe Hardware comes from, and the breakdown for each country.
Is it the same across PAL and Jp areas,... ie you don't get info from Game/gamestation here in UK but from other smaller companies?
Yeah... i've been tracking VGchartz for the last 2 months and enjoyed it. You guys are doin gr8 work, some people just dont understand how hard it is to track these statistics. It looks u have alot who so appreciate what ur doing :)
ioi, do you constantly get more retailers on the bandwagon?
Have you many in Japan? (You said just the one for your SMG numbers..)
Yeah I notice on some sites people say that VGchartz takes data from other sources (NPD) and then make changes, its really stupid but considering these are the same people who thought Saddam had WMD's i'm not surprised
Hmmm, patience ioi patience. Or I use the words of Jon Lucas: Believe! :o)
Good job ioi, give those lurking naysayers a verbal pounding!
....
When are the Japanese numbers going up?
ioi, sounds like you received a lot of Emails this week, or why the defense?
wow nice write up ioi, most of us core members know this, but yea this is really handy for the outside world. We could use the link of this to disprove many non-believers. Again, awesome =)
Essay Pro