Research manifesto Statistics Index CERG Home Page   CERG Resources Bibliography

Designing data collection devices Rolling your own versus tailor made One of the truly difficult tasks in effective social research is designing good data collection devices. Writing out a set of questions seems quite straight forward. But experience shows that the results do not always support this. Added to this is the problem is that there are few tailor made devices which fit the needs for a given research task. One of the best examples of this are student feedback devices. They have been designed to be generic to all courses. Even if the items are well written, they often fail to tap into the needs of particular courses. Devices which have been designed by others to cover projects similar to what you are looking at, are often only partly OK. If they are from another country the language may not be appropriate. So what do you do? The first thing you do is to make sure that you understand the language of the people you are dealing with. I don't mean that you have to learn Turkish, Mandarin, Serbo-Croat or whatever. What I mean is that you understand the level and approach to language of your subjects. Even students in higher education do not use many multi-syballic words. Neither do the majority use complex sentence structures whereby imbedded ideas are expressed in most sophisticated ways; ways which require high level language skills. The KISS principle applies in survey design - but do not talk down to you potential respondants. Open-ended versus closed-ended questions Device design is always plagued by the question of whether to use open-ended or closed ended questions. There is no doubt that richer data is obtained through open-ended questions but the trade-off is that it takes more time and effort to analyse the data. What you can do is to ask open ended-questions in a pilot phase for your project and to then convert questions to closed-ended, based on your content analysis, for the main data collection phase(s). Closed-ended and rating scales A closed ended question can be categorical, ordered or a rating scale (see Measurement scales). There are available quite powerful programs for analysising both categorical and ordered data. Not so many years ago, you had to stick with interval scaled data to be able to carry out powerful statistical analysis, but things have changed.