November 10th, 2004
Don't Believe Every Market Study You Read
It’s not thatI am not pro-VoIP. If I wasn’t, I wouldn’t be here.
But on days like today, when an IP "market study" crosses my desk that uses the general terms "were questioned" and "of the respondents,"I have some of my own questions about the methodology used.
My problem with thistechniqueoccurs when two things happen:the number of respondents is not specified, and the respondentsoverwhelmingly tilt toward a favorable reaction or prognosis. In technologymarketing, that often translates into a very high percentageof companies or people saying that "they will purchase" orwill "strongly consider purchasing" a (fill in the blank) product or solution in the next 12 months.
The problem is that there is a natural response bias. People who are energized about the product or solution being polled are most likely to answer. People who are not familiar with, or skeptical aboutthe product or solutionare not as likely to answer.
So don’t "not" believe the latest market survey you read. Just ask questions first. What percent of those who were surveyed responded? Whoconducted the canvass? Did the vendor do it themselves?Was the poll conducted bya reputable analyst firm? Was the firmpaid to do the study by a supplier with a vested interest?
Just lookin’ out for ya!
Russell Shaw is an enterprise computing journalist, analyst and author based in Portland, Oregon. See his full profile and disclosure of his industry affiliations.
