1. Would you do it yourself?
2 Does your survey pass the presentation test?
This is a good way of looking at things. If your survey was a presentation that you were delivering to a room full of 50 people how much more effort would you put into the design of it? I bet you probably would want to add a few more visuals for a start to liven it up. Where would you add these images? Would you change the flow of it to ensure it made sense? Would you trim back the text? Would your presentation be crammed with pages of dense bullet points? Now imagine you were presenting this to say 500 people or or even 1,000 presumably you would put even more effort into the design of the presentation? Well these are types of number of people who might well be consuming your survey so why not put the same effort into the design of it as you would a PowerPoint presentation.
3. Have you written the press release yet?
One of the best way of understanding what you really want to get out of the data generated from your survey is to write the press release summarizing its fantasy findings after you have drafted the survey. Its amazing when you start doing this what you focus on and what you leave out. All off a sudden half the questions in your survey might start to seem irrelevant. Its a brilliant way of refining and editing back your survey. This tips was given to my by one of my old bosses, Ivor Blight whilst working at Mirror Group newspapers and its been one of the most valuable pieces of survey design advice I have ever received.
4. Why are you asking that question?
Is it because it will produce a nice looking answer or because it is actually generating useful actionable feedback?
Take a customer feedback study where you ask your customers to rate you product or service. You find out that after polling 500 people that they score it a 4 out of 5. Now tell me apart from feeling pleased what are you going to do with this information to improve your product or service? What if instead you asked those 500 people to name ONE thing that might make your product or service better, how much more useful would that information be?
We also have a habit of making huge assumptions about about what a question will actually measure. A classic example would be the purchase intent question, "would you buy this new product?" This as I hope most of your reading this will be aware is proven to have little or no value as a predictor of sales. A far more predictive question would be to ask them if they think they would buy the product instead of the the main brand they buy.
I would challenge in particular you to consider the value of those banks of questions that so often get asked in surveys that attempt to measure brand characteristics like.. how much do you agree or disagree with these statements about this product... "its a modern brand", "its a trust worthy brand", etc. What are are these types of question actually telling you? Are you trying to find out the driving reasons why people buy a particular brands? Well why in that case don't you simple ask people that question "why do you buy this brand" . We did exactly this recently in an experiment to find out the driving factors behind why people purchase different brands of shampoo, there were over 50 clear reasons cited why people choose a particular brand ranging from the smell through to the size of the bottle, the impact of advertising, the type of ingredients, the appeal of the packaging and how well each shampoo cleaned different types of hair, some people don't believe there are any differences in one shampoo to another so buy the cheapest some people buy shampoo because it was recommended by their hairdresser, others chose a brand because they liked the cream feel of the shampoo or the way it lathered up or they thought it was more ethical, but only 3 people out of a sample of 500 said they chose their brand because the felt it was modern that is 0.6%.
5. Have you tested the survey?
And I don’t mean for routing errors and spelling mistakes. I mean have 30 people done your survey and have you had a really good look at the data to see what it is delivering and how useful it is and what its missing and what could be improved? So few people in my experience properly pilot their research studies or use piloting as a means to develop and improve their survey and yet this in my opinion is the single most effective way of improving your market research.
This has to be the key question you should ask yourself. If you were sent your survey by someone else would complete it? Would you give it your full attention to every question? If the answer is no, then in that case don't expect the average respondent to answer your survey properly either.
2 Does your survey pass the presentation test?
This is a good way of looking at things. If your survey was a presentation that you were delivering to a room full of 50 people how much more effort would you put into the design of it? I bet you probably would want to add a few more visuals for a start to liven it up. Where would you add these images? Would you change the flow of it to ensure it made sense? Would you trim back the text? Would your presentation be crammed with pages of dense bullet points? Now imagine you were presenting this to say 500 people or or even 1,000 presumably you would put even more effort into the design of the presentation? Well these are types of number of people who might well be consuming your survey so why not put the same effort into the design of it as you would a PowerPoint presentation.
3. Have you written the press release yet?
One of the best way of understanding what you really want to get out of the data generated from your survey is to write the press release summarizing its fantasy findings after you have drafted the survey. Its amazing when you start doing this what you focus on and what you leave out. All off a sudden half the questions in your survey might start to seem irrelevant. Its a brilliant way of refining and editing back your survey. This tips was given to my by one of my old bosses, Ivor Blight whilst working at Mirror Group newspapers and its been one of the most valuable pieces of survey design advice I have ever received.
4. Why are you asking that question?
Is it because it will produce a nice looking answer or because it is actually generating useful actionable feedback?
Take a customer feedback study where you ask your customers to rate you product or service. You find out that after polling 500 people that they score it a 4 out of 5. Now tell me apart from feeling pleased what are you going to do with this information to improve your product or service? What if instead you asked those 500 people to name ONE thing that might make your product or service better, how much more useful would that information be?
We also have a habit of making huge assumptions about about what a question will actually measure. A classic example would be the purchase intent question, "would you buy this new product?" This as I hope most of your reading this will be aware is proven to have little or no value as a predictor of sales. A far more predictive question would be to ask them if they think they would buy the product instead of the the main brand they buy.
I would challenge in particular you to consider the value of those banks of questions that so often get asked in surveys that attempt to measure brand characteristics like.. how much do you agree or disagree with these statements about this product... "its a modern brand", "its a trust worthy brand", etc. What are are these types of question actually telling you? Are you trying to find out the driving reasons why people buy a particular brands? Well why in that case don't you simple ask people that question "why do you buy this brand" . We did exactly this recently in an experiment to find out the driving factors behind why people purchase different brands of shampoo, there were over 50 clear reasons cited why people choose a particular brand ranging from the smell through to the size of the bottle, the impact of advertising, the type of ingredients, the appeal of the packaging and how well each shampoo cleaned different types of hair, some people don't believe there are any differences in one shampoo to another so buy the cheapest some people buy shampoo because it was recommended by their hairdresser, others chose a brand because they liked the cream feel of the shampoo or the way it lathered up or they thought it was more ethical, but only 3 people out of a sample of 500 said they chose their brand because the felt it was modern that is 0.6%.
5. Have you tested the survey?
And I don’t mean for routing errors and spelling mistakes. I mean have 30 people done your survey and have you had a really good look at the data to see what it is delivering and how useful it is and what its missing and what could be improved? So few people in my experience properly pilot their research studies or use piloting as a means to develop and improve their survey and yet this in my opinion is the single most effective way of improving your market research.