Join the Network to Access Resources

Join the Network

Already Joined?

Already joined?

Constructing Surveys

Tips for designing effective surveys of interfaith work

Constructing Surveys: Tips for Interfaith Educators

When thinking of interfaith assessment, many college educators imagine surveys and questionnaires. Whether brief questionnaires to collect snippets of feedback after an educational experience or longer forms to understand multiple perspectives on department services, surveys can be used in a range of contexts and for various purposes to advance interfaith work.

Is a Survey Your Best Option?

When deciding whether to use a survey, determine whether it is the most appropriate tool to get the feedback you need. Surveys should never be done just because they seem like the easiest path to interfaith assessment. Poorly constructed surveys can produce misleading and/or perplexing results. If you decide that a survey is the best option for capturing information you need, then we recommend you collaborate with assessment or institutional research offices for support constructing meaningful and sound survey questions. Particularly for longer and/ or perennial surveys, these professionals can provide insights and tips that will save you from confusing and potentially less useful results. However, if this is not feasible, consider the following tips and tricks to create a survey that collects quality information.

The Basics: Organizing the Survey

Creating quality surveys begins with organizing the survey. Lack of forethought and planning often leads to less effective surveys. Below are a few tips to help you start constructing your survey.

  • Put questions for information you need near the beginning – Particularly for longer surveys, you may want to make sure the questions that collect the most important information are near the beginning. If a student gets distracted or stops taking the survey for some reason, you will have responses to the questions that are most important to you.
  • Group questions by topical areas – You should carefully consider what types of questions belong together and how you want to organize groups of questions. This allows you the opportunity to provide good instructions for different sections and prevents confusion if students feel they are hopping around between different topics.
  • Start with less threatening questions – If you have questions that students may perceive as more sensitive or questions that elicit charged feelings, you may want to put these questions toward the end of the survey.
  • Avoid excessive questions/items – Only ask questions for information you need. Student are surveyed regularly and many campuses suffer from “survey fatigue” (a phenomenon where students become apathetic to surveys and increasingly decline to participate in them). After creating a survey, always review it, and evaluate whether you need each question or whether you can get information from somewhere else.
  • Keep open-ended questions to a minimum – When used appropriately, open-ended questions can provide highly valuable, nuanced information. But open-ended questions require more mental energy from students and can be overwhelming if there are too many of them. Thus, open-ended questions may be best near the end of survey in case a student decides to stop taking the survey because they do not want to think through a response.

Dos and Don'ts for Writing Questions

In this section we provide the dos and don’ts of survey construction. For each of the dos and don’ts we provide an example of a quality survey item and a poorly constructed survey item. This list is by no means exhaustive, but it is a good start to collecting quality information about your students.

Do:

  • Do include examples for terms or concepts that are not clear – Sometimes surveys about interfaith leadership and other complex topics can include terms that are not easily understood by the survey participants.

Incorrect

During the fall semester, how often have you participated in interfaith service projects:

Never (1) Rarely (2) Sometimes (3) Frequently (4)
       


Correct

During the fall semester, how often have you participated in interfaith service projects (e.g., volunteering with religiously diverse peers):

Never (1) Rarely (2) Sometimes (3) Frequently (4)
       

By clarifying the terms, you help to ensure more accurate responses.

 

  • Do provide specific time frames when asking students to recall past actions – Sometimes interfaith educators want to know how often students engage in interfaith behaviors

Incorrect

How often have you participated in interfaith service projects (e.g., volunteering with religiously diverse peers):

Never (1) Rarely (2) Sometimes (3) Frequently (4)
       


Correct

During the fall semester
, how often have you participated in interfaith service projects (e.g., volunteering with religiously diverse peers):

Never (1) Rarely (2) Sometimes (3) Frequently (4)
       

The more specific and contained the timeframe, the more accurate the information you get will be.

 

  • Do make sure responses are exhaustive and mutually exclusive – Response options should cover all possible responses and should be mutually exclusive.

Incorrect

To what degree were your interfaith dialogue experiences enjoyable:

Enjoyable (1) Frustrating (2) Tense (3) Funny (4)
       


Correct

To what degree were your interfaith dialogue experiences enjoyable:

Not enjoyable at all (1) Somewhat enjoyable (2) Enjoyable (3) Very Enjoyable (4)
       

In the incorrect example, note that it would be impossible to list all potential descriptors for that experience and many descriptors may overlap in meaning. In the correct example, the responses are exhaustive and mutually exclusive.

 

  • Do label rating scale values and assure intervals between response options are relatively even – If you provide a rating scale of 1 to 4 for “frequency of participation in interfaith service projects” responses, the frequency options should be labeled. The frequency options should be incrementally spaced. This means that responses should not radically change from one option to the next.

Incorrect

During the fall semester, how often have you participated in an interfaith dialogue:

Never (1) Often (2) Frequently (3) All the time (4)
       

 

Correct

During the fall semester, how often have you participated in an interfaith dialogue:

Never (1) Rarely (2) Sometimes (3) Frequently (4)
       

In the incorrect example the scale jumps from never to often, a large gap. In addition, there is little difference between often, frequently, and all the time. There should be consistent, incremental movement from one response option to the next. In the correct example the options move incrementally from never to frequently. The distance between each item is similar and appropriate.

 

  • Do pay attention to bias – The way in which a question is framed or the instructions are written may lead students toward particular responses. When writing questions, double-check that questions, instructions, or the order of questions does not guide students to particular responses.

Incorrect

Please rate how inadequate the following facilities are:

Very inadequate (1) Somewhat inadequate(2) Somewhat adequate (3) Very adequate (4)
       


Correct

To what degree do you agree or disagree with the following statement: The campus interfaith room is adequate for our organizational needs.

Strongly disagree (1) Disagree (2) Agree (3) Strongly Agree (4)
       

The incorrect example creates a scale that is biased toward a negative reaction: the highest rating is fair, limiting students’ ability to express positive reactions. Furthermore, it also assumes the inadequacy of the facilities in the question instructions. The correct example does not presume the student has a particular stance.

 

Don't:

  • Don’t forget to provide instructions – Do not assume students will inherently know how to complete your survey. Including succinct and clear instructions will help to assure quality responses.

Incorrect

40 Days of Peace: An Interfaith Vigil:

Never Rarely Sometimes Frequently
       


Correct

Please rate your overall experience of the following event: 
40 Days of Peace: An Interfaith Vigil

Poor (1) Fair (2) Good (3) Excellent (4)
       

In this example students would not have understood that the number one was associated with a poor score. Providing clear instructions to students will help you collect the information you are seeking.

 

  • Don’t use jargon, complex sentences, and/or charged language – Good survey questions are often brief and direct. Use language that is not highly charged and words that all students will be able to understand.

Incorrect

Please rate how terrible outsourced campus dining options are at accommodating the dietary restrictions of religious minorities:

Poor (1) Fair (2) Good (3) Excellent (4)
       

Please rate how terrible outsourced campus dining options are at accommodating the dietary restrictions of religious minorities.

Correct

Please rate campus dining accommodations for people with dietary restrictions (e.g., vegetarian, vegan, kosher, halal, etc.):

Poor (1) Fair (2) Good (3) Excellent (4)
       

By calling the dining options terrible, the incorrect example uses charged language that is likely to inappropriately produce biased responses. Also, the incorrect question uses unexplained and unnecessary jargon (see: “outsourced” and “religious minorities”). The correct example uses clear, straightforward language.

 

  • Don’t use “two-pronged” questions – This is a common pitfall in survey writing. Whenever you see a survey question with “and” or “or” in it, you should make sure it is not asking the student to respond to two different questions.

Incorrect

To what degree do you agree or disagree with the following statement: The interfaith luncheon was insightful and informative.

Strongly disagree (1) Disagree (2) Agree (3) Strongly agree (4)
       


Correct

To what degree do you agree or disagree with the following statement:

  Strongly disagree (1) Disagree (2) Agree (3) Strongly Agree(4)
The interfaith luncheon was insightful.        
The interfaith luncheon was informative.        

A student could have thought the luncheon to be insightful, but not informative (or vice versa). Twopronged questions create problems for students when responding and for educators when trying to make meaning of responses to the question.

 

Conclusion

Because we know interfaith educators may not have a background in survey methodology but are already conducting surveys with their students, we created this resource to provide a few key tips that will improve the quality of surveys being administered on campus. We do not intend for it to be a comprehensive guide for survey construction. However, we do believe that applying these tips can put educators on the path to creating effective surveys that will give them information that improves interfaith efforts. Many of the tips shared here were adapted from Newcomer and Triplett’s chapter titled “Using Surveys”. To explore the topic further, educators are encouraged to read the chapter as it provides a more in-depth review of surveys.