Sneaky surveys (and push polls)
First thing: All open access online surveys are essentially inaccurate, because the group that takes the time to answer the survey is usually different from the general public.
Second thing: Don’t confuse a survey with a census. A survey asks a randomized but representative group some questions and then seeks to extend the answers to the entire group as a whole. A census seeks to ask everyone in the group, so that no generalization is required.
In general, you don’t need a census. What you need is a correctly representational group, which can be dramatically smaller than the entire population. The huge mistake is believing that you need to survey more and more people. You don’t. And your work to reach more people actually makes your survey less accurate not more (see the first thing). And yes, this is true even if you’re a solo creator wondering if your novel is any good.
In the age of good stats, the best use of a census is to establish a 1:1 relationship between what someone feels and who that person is. Asking every single person at a restaurant what they want for dinner is a census, a useful one, because you can then serve each person exactly what they want for dinner.
Third: You might believe the survey someone just emailed you to fill out is anonymous. It probably isn’t. Check out this explanation from Survey Monkey. It turns out that tracking by IP and even email address/name is a built-in feature. If you get a survey link by email or even as you browse a site, it’s a safe guess to imagine that your answers are tied in some way to your other interactions with the organization that posted the survey. Respondent beware.
And fourth: Asking someone a question can change the way they feel. Done crudely, this is called a push poll (“Did you know that Bob was indicted last year?”) but even asking someone a thoughtful question about their satisfaction can increase it.
Okay, two more things:
At the conclusion of the endless surveys when they ask you if you have anything else to add, don’t bother. It’s not like the CEO is busy reading your comments.
The single best way to figure out how people feel isn’t to ask them with some focus-group survey. It’s to watch what they do when given the choice. “This or that?” is a great way to get to the truth of our preferences.