My company conducts surveys on behalf of customers, but personally and professionally, I think they suck!

Dont get me wrong. Wanting to get your customers opinions is a good thing.

I just dont think surveys get at peoples opinions efficiently or effectively, and there are better means at our disposal.

For example, last night at the checkout counter at the supermarket I picked up a survey. It has a listing of eleven items, ranging from the speed and accuracy of the checkout to the freshness and selection of meat and fish. Plus, there are spaces, tiny lines at that, for recording comments and suggestions.

Its a lot to cram onto a 4 by 7 pre-stamped card.

Anyway, here are just five of my concerns about a survey of this type:

(1) Only a fraction of shoppers will see it in its plastic bin, pick it up, and take it with them. So, the response wont be representative of shoppers at large, because many just wont see it or know what it is. One card should be placed in the hand of every shopper, or in her shopping bag.

(2) By asking that the form be mailed back, the company is precluding people from noting their reactions just as theyre occurring. They will be in an altered frame of mind when they get home, or a day or two or even a week later when they fill it out. Lots of other variables will intrude into their thinking, making their responses suspect. If the company wants genuine responses, they should provide pens at the checkout counter, so cards can be filled in right away. But, if they offer pens, people will take them! So, faced with a choice between investing in pens, and getting less fresh and valid replies, the company is choosing to save money on pens.

(3) There is no control on how many forms a given shopper can take and fill-out. If you have real heartburn with a clerk, you can flame them by sending back ten scorching cards. Or, you can put halos over your friends heads, if you wish. In other words, there is no control against stuffing the ballot box, jeopardizing the validity of the overall response.

(4) Checkers can differentially discourage or encourage shoppers from taking and filling out the cards by frowning or smiling as people reach for them. Again, this will skew the response.

(5) The shoppers at this store are less likely to participate because many hail from formerly totalitarian countries where you would be punished for revealing your true feelings. The tacit cultural code could be stated this way: dont volunteer, and dont bring attention to yourself, or else!

The mailer is addressed to the President, and thats a nice touch, but if hes genuinely interested in knowing how his customers feel he should get out of his office and visit the stores, himself, and observe them. Or, he can achieve a similar outcome by commissioning an unobtrusive study of customer responses, or by using mystery shoppers, and even focus groups.

Then again, were not sure of his genuine motivation, are we? He may just want his survey to give the appearance that hes customer focused.

In that case, hes wise to save money on pens.