ANTH/MACS/SOC 255: FROM RESEARCH DESIGN TO SURVEY CONSTRUCTION
ASSIGNMENT #2: SURVEY CONSTRUCTION (30%)
ACTIVITIES: Design a survey to test your hypothesis from the model you created in assign#1.
1. Write questions for all your variables to test your hypothesis from the model you created in assignment 1 (or revised model).
2. Include important background demographic variables.
3. Make sure you provide a sufficient number of questions to thoroughly measure all the variables in your model. The number of questions per variable will depend on their complexity.
4. Include at least one matrix question for one of the variables in your model.
5. Include at least one open ended question that is crafted for maximum effectiveness of variable measurement (“other—please specify” is often necessary on a survey, but is not sufficient for this requirement).
6. Refine your questions.
This refers to both their content and their physical structure. Follow the principles described in the text and in class to make sure you avoid problems such as bias, emotionally charged language, double-barreled questions, etc.. Make sure you have clear, precise, language, mutually exclusive and exhaustive categories in closed-ended questions, and a reasonable range of answer categories, amongst other issues.
7. Organize your questions
Make sure your questionnaire is formatted so that it is easy to read and easy to record responses. Make sure you include instructions, such as whether your participant should circle one answer or check as many as apply, and what to answer next if a question does not apply to them (e.g. contingency questions). Provide organizational ‘niceties,’ such as section headings, descriptors, etc.. Sometimes you may want to add a few questions that help make the questionnaire flow better (i.e. “filler” questions that help lead the respondent from one type of question to the next).
8. Pretest
Print out 3 questionnaires and find 3 volunteers (friends, family members) to try the questionnaire. **Note: they don’t need to answer honestly; you do not have ethics clearance to actually conduct your research.** The purpose here is to “test” the questionnaire itself: its wording, formatting, etc., to see if it makes sense to someone else, and if they can follow your instructions, etc.. Ask your volunteers to write comments. Revise your survey in response to your pretest results.
(***SEE PAGE TWO FOR MORE INSTRUCTIONS***)
WRITTEN REPORT—FOLLOW THIS FORMAT EXACTLY.
A. Explanation: (400-450 words)
Provide a brief explanation of your model and your hypothesis, indicating its importance, and the importance of your selected variables. Make sure the nature of your variables, how they are operationalized, and your causal logic are crystal clear (what causes what and why). Re-use, with appropriate revisions, whatever of your assign# 1 is useful to you. Underline your hypothesis.
I will assume some form of probability sample for your survey unless you indicate (and justify) otherwise. Indicate your population and study population.
Attach a copy of your model/diagram.
B. Evaluation: (150-200 words)
This is your opportunity to critically evaluate your questionnaire as a measurement instrument. You can either do the below for all 5 of your variables, or focus on one or two to treat in depth. All measurement instruments have flaws; I am looking for your ability to be reflexive and demonstrate an ability to critically evaluate the choices you make. Focus on what provides the most depth and insight (e.g. telling me you forgot to make a check box for one of your closed-ended questions and thus corrected it after the pretest can go without saying). Things you can reflect on:
(i) The conceptualization of your variable: If the variable is complex or could have been conceptualized differently, are there any consequences to how you conceptualized it?
(ii) Your questions: Were you able to capture the essence of your concept(s) with your measures? Are there any qualifications you should make (e.g. acknowledgements of unavoidable weaknesses in the research)?
(iii) Pretest: Did any substantive problems arise in the pretest? If so, what was their nature? Why did you make the changes that you did in response?
C. Table.
Provide a table to indicate which questions provide data for which variables. Make sure you clearly indicate the question #s and which variables they are intended to measure, which are filler/transition questions, demographic questions, etc. (see example below).
Variable Role in Model Question #
“Studiousness of significant other” Independent Variable Q7, Q8, Q9a-g…
“Employment level” Independent Variable Q12, Q13…
“Study habits” Intervening Variable Q14, Q15, Q16…
“Academic success” Dependent Q19, Q20, Q26…
“Gender” Demographic Q1
… Filler Questions Q5, Q6
… etc… …
D. Appendix
Attach a copy of your final questionnaire, your three pretested questionnaires, your model, and anything else I may need to evaluate your project.