Getting access to good quantitative data in education research, such as test scores, can often be problematic for a variety of reasons. This kind of empirical data can be very useful for demonstrating impact, for example that the introduction of open textbooks on a course led to an improvement in overall performance of the cohort. However, you may be unable to get this sort of data, or be interested in other issues, such as perceptions of content, attitudes to OER, understanding the context, etc. For this type of research, surveys can be very useful.
The OER Research Hub used and openly released a set of 54 core survey questions, which comprised of demographics, teacher and learner specific questions and questions specifically aligned to each of the eleven hypotheses that underpinned the project’s research.
Selected hypotheses were aligned to individual research collaborations and provided a consistent framework for the project’s research activity. This common bank of questions therefore enabled comparison across different user groups, e.g. teachers and learners in both non-formal and formal contexts and across different sectors (e.g. higher education, college or further education, informal learning and school or K12). You can review the reusable and remixable, CC BY licensed, question bank (a copy is also available in the Appendix).
Some hypotheses were applicable only to specific collaborations whilst in some instances hypotheses were better answered through looking at other data sources or by reviewing selected responses to questions (e.g. demographics and other questions).
A quick checklist of things to consider when designing your survey:
- Are your survey questions answering your research question(s)?
- Is a survey the best way of gathering evidence for your research question(s)? Data for the OER Hub’s Hypothesis C (“open education models lead to more equitable access to education, serving a broader base of learners than traditional education”) was better addressed by combining demographic information (the “broader base of learners”) with responses to questions for Hypotheses A and B, for example;
- If you’re planning to carry out further research, e.g. interviews, will you be using the survey as a way of identifying potential volunteer participants? If so you will need to add in further questions at the end of the survey.
- Ensure each question counts and think of your respondents… keep the survey as short as possible!
- Don’t ask people irrelevant questions;
- Don’t forget appropriate ethical clearance when planning your survey (see the Ethics chapter) and provide an opening statement at the start of your survey to advise potential participants on the survey’s purpose, how long it will take to complete, how you will use and store survey data, etc. We’ve provided an example from the project below.
There are several reasons why you might want to repurpose existing survey questions. The questions used and devised by the OER Research Hub were informed by nine existing surveys on open resources and practices. You can find out more about these surveys and the background to survey question creation here.
- One benefit of reusing existing survey questions if that they have already been tested ‘in the field’ and refined accordingly;
- Reusing questions can save you time rather than ‘reinventing the wheel’;
- Reusing survey questions might enable you to generate comparative data in instances where a corresponding dataset is openly available. For example, the OER Hub data set is openly available.
Using the OER Research Hub survey questions
Before you review the OER Research Hub questions it is important to be clear about the purpose of your research. This will help you identify questions that you would like to reuse.
Here is an overview of the types of questions provided by the OER Research Hub:
| No(s) | Type of question | Aim of question(s) |
| 1_1 – 1_9 | Demographic questions | Basic information about participant |
| 1_10 | Question regarding ways in which participant has accessed internet | Find out how and where participant usually accesses internet |
| 1_11 | Question noting a range of online activities that participant may have participated in or done in last year | Insight into range of activities and level of digital literacy of participant |
| 1_12 – 1_14 | Questions for learners in both non-formal and formal contexts | Find out more about why, where and how learner studies |
| 1_15 | Question asking participant why they use a particular resource/website etc. | Ascertain reason for using (in this instance) a particular open resource |
| 1_16 – 1_21 | Questions aimed at educators, including subjects taught and those where OER is used, open practices, length of teaching etc. | Insight into educator practice |
| 2_1 – 2_2 | Questions for Hypothesis A “Use of OER leads to improvement in student performance and satisfaction” | Likert scale questions regarding the perceived impact of OER on students (for educators) and perceived impact on formal studies (for students) |
| 3_1 – 3_15 | Questions for Hypothesis B “The open aspect of OER creates different usage and adoption patterns than other online resources” | This set of questions begins with the Hewlett definition of OER. These questions should be used prior to any from Hypothesis A (see note below) and cover use, type and reason for using OER, challenges when using OER, open licensing and sharing practices |
| 4_1 – 4_2 | Questions for Hypothesis D “Use of OER is an effective method for improving retention for at-risk students” | As the spreadsheet narrative notes there are better ways of answering this hypothesis than via a survey. However two questions for educators on perceptions of at-risk students and the impact of OER have been included |
| 5_1 | Question for Hypothesis E “Use of OER leads to critical reflection by educators, with evidence of improvement in their practice” | A question for educators to self-access the impact of OER on their own practice |
| 6_1 – 6_3 | Questions for Hypothesis F “OER adoption at an institutional level leads to financial benefits for students and/or institutions” | Questions for both learners and educators regarding OER cost savings |
| 7_1 | Question for Hypothesis G “Informal learners use a variety of indicators when selecting OER” | Question to ascertain what features of a resource are most important to users (aimed at informal learners and but can be used with educators too) |
| 8_1 | Question for Hypothesis H “Informal learners adopt a variety of techniques to compensate for the lack of formal support, which can be supported in open courses” | Question to ascertain what techniques learners use to support themselves in their learning when not within a classroom context |
| 9_1 – 9_3 | Questions for Hypothesis I “Open education acts as a bridge to formal education, and is complementary, not competitive, with it” | Questions regarding likelihood of doing certain activities as a result of using OER and whether use of OER by formal learners prior to signing up for a course influenced their decision |
| 10_1 | Question for Hypothesis J “Participation in OER pilots and programs leads to policy change at an institutional level” | Question asking educators whether their institution has an OER policy |
| 11_1 – 11_2 | Question for Hypothesis K “Informal means of assessment are motivators to learning with OER” | Question to both ascertain what features learners and educators have encountered when using open resources and what they would find motivating |
It is important to note that although the hypotheses are listed in order, the questions provided are not necessarily in the order you should use them in. For example the definition of open educational resources (OER) that is provided at the start of the Hypothesis B questions should be provided before any questions from Hypothesis A (where the term OER is used but not defined) or those that use the term ‘OER’ are used. When you are designing your survey ensure that you introduce definitions at the appropriate point in your survey.
It is important to capture some information about your participants that will enable you to ‘slice’ the data in interesting ways and make claims about different groups of users. For example, do you want to make comparisons between different age groups or between participants from different parts of the world? Demographic questions will help you obtain relevant data and thus has a crucial role to play in your survey. Question 1_1 to 1_9 in the question bank provide some basic demographic questions that you might want to consider using at the start of your survey.
Before you ask your participants some basic information about themselves, you will need to provide some information about the survey itself for your potential participants. This will help them decide whether they want to participate in your research. An example OER Research Hub survey (the 2014/2015 OpenStax College student survey) is shown below, with a narrative (in red) explaining what each component of the text seeks to achieve.
Once you have introduced your survey and asked some basic demographic questions, you can begin to structure your survey. As noted earlier, be careful about the ordering of questions. At the end of the survey you might want to ask for volunteers for follow-up research (describe what this might be and ask for relevant contact details) and thank people for their participation. Where possible, provide a URL to your project or personal website for people to find out more about your research.
When reviewing the question bank you might want to change the language used in a question or add other options into particular questions. During the OER Research Hub project and because of the collaborative nature of our work, it was often the case that questions were tweaked slightly. This is not in and of itself a problem but you do need to consider how much (if at all) this will weaken any comparisons you might want to make with existing data for these questions.
Finally, as good practice and in the spirit of openness, consider sharing how you used the questions and what your research findings were with the wider community and with us.
