Building a Questionnaire – Why My Survey Stinks

Why do people ask such repetitive and stupid questions?

Simply put, questions are hard, answers are easy – coming up with a good question and being able to do something with the answer is far harder than one might think.
This will be a two-part post in which I cover a case study of why what I am doing with my Knowledge Management Survey questionnaire is wrong, why recruiters ask so many dumb questions, and why people ask questions that have already been answered many times before.

More usefully though, these blog posts will guide you through a process of how to ask questions and design questionnaires that give you usable answers, lead to better questions, and make you look smart.

Firstly, my Knowledge Management survey.

I have many books on Knowledge Management, in fact they occupy two entire shelves, and several have questionnaires to scope out all kinds of things about Knowledge Management adoption, climate, culture, etc.
In fact, I have shamelessly re-used them (as any good KMer should) over the years and had pretty good results.
However, it always bothered me that not all the question items made sense, some seemed to break question rules (like being double-barreled), and they didn’t cover all the areas I wanted to know about.
Plus of course, I wanted my own.

The problem though, is although I have the questions, and these are doubtless the work of clever and experienced people, I don’t have the handbook that they must have created in order to generate the questions – I only have the questions.
You heard right, a questionnaire isn’t a matter of banging out some good-sounding questions, checking for spelling and launching.

The Right Way

You start by creating a handbook (Oppenheim 1998)

In fact you start with an initial Problem Area description, then progress to considering content and consider the Mental Model, Literature & Experience, and Process & Outcomes, and then formulate a Research Problem Statement.

Once that is securely tidied away, you move on to what the specific Research Question is, its Paradigm, applicable Research Method and Context, (Swanson 2005) and once you have that in hand, you will know if this is best done as qualitative or quantitative research.

If you decide a questionnaire is the best approach, you go through a whole other process that entails identifying the Constructs, determining the target population, and the survey methodology.
Then you bang out loads of ideas about the dimensions of the constructs and come up with candidate questions – questions which must obey a whole slew of criteria

For instance:

  1. They must be in active voice
  2. They must be collectively exclusive and exhaustive
  3. One question, one concept – no double-barreled questions (so avoid “or”, “and”, “therefore”, “either”, “both”, etc.)
  4. No Slang
  5. No loaded language or leading questions
  6. Avoid negative statements
  7. Agent of action must be clear
  8. Etc.

(Oppenheim 1998; Collins, du Plooy et al. 2000)

Collection and Analysis are two subjects all of their own, just like sampling techniques.

The “Easy” Way

You do like I did, you grab nice questions from elsewhere (trusted sources of course), create a questionnaire wiki, like mine, invite a bunch of people to help for free (which I am doing with you), and then add an explanation of what each question is intended to measure. Put the questionnaire online, like I did, and invite people to take the survey (like I just did) and to tell you if anything didn’t make sense and how long it took.
Then you start rewording and deleting or splitting questions so they obey the rules above.

At this point you can write a blog post on your questionnaire and invite people to recruit others to help, and tell your readers that when the questionnaire is finished, they will gain greatly by being able to use the survey for their own purposes and at their own firms, and of course, you can emphasize how this makes it look like they are members of a big Community of Practice.

Next week I will discuss how you helped.

Until then, happy Thanksgiving and remember to eat plenty of greens and to drink one glass of water for each glass of beer or wine.

Bibliography

Collins, K., G. du Plooy, et al. (2000). Research in the Social Science. Pretoria, University of South Africa.

Oppenheim, A. N. (1998). Questionnaire design, interviewing and attitude measurement, Pinter Pub Ltd.

Swanson, R. A. (2005). Research in organizations: Foundations and methods of inquiry, Berrett-Koehler Publishers.

 

Please contribute to my self-knowledge and take this 1-minute survey that tells me what my blog tells you about me. – Completely anonymous.

~~~

Matthew Loxton is a Knowledge Management professional and holds a Master’s degree in Knowledge Management from the University of Canberra. Mr. Loxton has extensive international experience and is currently available as a Knowledge Management consultant or as a permanent employee at an organization that wishes to put knowledge to work.

 

Advertisements

Tags: , , , ,

One Response to “Building a Questionnaire – Why My Survey Stinks”

  1. Tweets that mention Building a Questionnaire – Why My Survey Stinks « Matthew Loxton's KM & OL Blog -- Topsy.com Says:

    […] This post was mentioned on Twitter by Roxanne Persaud, Matthew Loxton. Matthew Loxton said: why company surveys often stink, part one http://wp.me/pLiGk-9v […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: