Transcripts for Open Research 1.3: What does open research mean to others?
Chris Pegler on the Difference between Open and Traditional Research
Well, for me, when I tend to talk to people about open I use the libre and gratis sort of qualifiers. So in terms of research being open access I think that this is obviously important to researchers and not just to researchers but people… public members who are doing research. It means that they can have access to high quality content. Now if I can take myself as an example, I’m a national teaching fellow, I was recently at a symposium with other national teaching fellows – we’re all still active to some extent in education – but if you are not tied to an institution anymore you don’t have access to institutional libraries. So you can see that for people like that then access to open access educational resources is important. So that’s the sort of libre side of things.
In terms of the gratis, I think this is fairly untested with open research. Whereas we’ve been sort of getting our hands dirty with open educational resources in terms of being able to remake them and remix them. I’m not convinced I’ve really seen it. Only in terms of the release of open data have I seen this sort of happen in open research and I think it’s important. It has the potential to become very important. But I’m not sure how you resolve this issue about ownership and trust, and if you’ve carefully worded a research journal article whether you’d be happy with someone taking that and reworking it.”
Patrick McAndrew on the Difference between Open and Traditional Research
Well I think when you are researching open educational resources there are some additional challenges that come about it. I think it’s reflected in the fact that over the last 10 years of open educational resources there have been a lot of projects that haven’t really recognized the need for research. Which is fine, you want to get on and do things, being open can seem like a no brainer, you just make it open and free so it must help people. But it does mean that we don’t necessarily know what’s helped us in the past. And I think one of the challenges is a lot of work going on without the space for the research.
Another one is the openness. The openness means that your content can reach out to people but you don’t necessarily know who those people are. You don’t even know if they’re using material directly from your site, or if it’s gone and travelled and ended up somewhere else. If you do something in a classroom then least you know where your students are. So when you are in the open you don’t necessarily know.
So there’s a flipside though, I think, that along with these challenges there’s ways to tackle them. And as long as you stick to sound principles in the research – so that’s where the connection comes back to other primacies of research, if you treat ethics well but make sure that it gives you the space to get the data you need, if you set out some good research questions – hypotheses – then openness gives you a lot of scope to spot places to get this data. So I think following sound research principles, realizing that perhaps you’ve got to be quite clever in how you apply them, then I think open research can connect back with other research methods.”
Cheryl Hodgkinson-Williams on the Difference between Open and Traditional Research
Yes and in fact we’ve made a statement and we have a principle that we are wanting to uphold and that is to make our research open. So what we mean by that is that rather than waiting right until the end of a research project, where we have the findings and we publish that in an open access journal, what we are hoping to do is to make each stage of the research cycle open. That doesn’t mean to say that we’re going to have everything up because clearly research is messy and people don’t have time to go through every little bit of messiness, so we will select. But for example the fact that our proposals are available, we’re going to make our literature reviews available, our conceptual frameworks – why we chose what we did and alternatives – we’ll make that available, our methodologies that we undertook – we’ve got quite a variety – and then how we went about our instrument development process for surveys, questionnaires, focus groups, document analysis… So not only sharing the final survey instruments, or any of the instruments, but in fact shining a light on some of the underlying assumptions that we’ve made so there’s a much clearer understanding about why those particular questions were chosen. We will then – and this is our biggest challenge – is to actually make our data open. And that I’m less confident about at the moment because we haven’t done it yet, but that’s our intention. But in terms of our initial dissemination, we have an idea that instead of waiting for the final report that we’ll be creating a dynamic research log and building up the research as it goes along, in a way which the report structure is layered. So you can get a summary and then you can get a summary of the projects but then if you want to delve deeper you can actually go into the actual questionnaires used in that particular project, the actual data, the list of references, the people who were involved, their contact details.
And this obviously will be dynamic in the sense that people can leave comments and ask questions on the actual document itself. Or for example challenge some of our assumptions. So we’re hoping in that way to make it slightly more dynamic and interactive than before. So these I must say – caveat – are intentions. I haven’t seen, apart from the OER Hub, which are making quite a lot of the data, the visualisation of the data available – which we are obviously going to connect into as well – I haven’t yet seen a research project, certainly not in OER, there might be in other fields, where the entire research process is quite as transparent as this one.”
Martin Weller on the Difference between Open and Traditional Research
Yeah, I think there is. So open research for me, I think you can see it more or less a list of different things, you needn’t necessarily do everything on that list but there are different approaches you might take to open research. So open research would involve possibly using open methodologies, so maybe you’re using crowdsourcing. Or it might be looking at the availability of openly available data, doing social media analysis, those kind of things.
It might be a methodological approach to open resource. But more commonly I think it’s about opening up the research approach itself, so as you’regoing along the process you’re being much more open about what you’re doing. Whereas traditionally, I think, the way we think about research has been much more about you get your research funding, you do a 2 year research project, and then maybe publish an article at the end of it which may be in a proprietary journal. Whereas open research is much more about having all of that process open, all the way along. So maybe even before you’ve got your research grant you’re blogging ideas, you’re tweeting ideas, getting input from other people. And as you’re going along through your project – assuming it is a project – you are doing things like blogging your results, putting things out there, you’re sharing data. So open data is very important … as long as it’s OK to share and can be anonymised and shared openly with other people. And, importantly, you are publishing any findings in open access journals. So there’s a kind of a theme, a strand of openness, than can run through research, I think.”
Patrick McAndrew on where open has made a difference to his research practice
OK. Well, I think in our own research practice, one of the things openness has led us to do is explore benefits that perhaps weren’t on our original expectations much more. So openness doing resources that leak out and go round the edges, made us spot that there are technology advances that can be sparked by openness, there are certainly new routes in for students and learners that come from having open materials. There are new connections to be made. So it’s led to a position where we see open research as a very collaborative process, making new connections with people, being able to apply surveys more broadly perhaps. That’s the advantage of the openness coming in, there’s are other groups doing interesting things as well. So I think openness has led us to spotting some less obvious results and to working in a very collaborative and open way itself. So open research becomes part of an approach to being a member of the open resource community as well.”
Chris Pegler on where open has made a difference to her research practice
…One of the things I would say, I’d make a point about, is that I see myself primarily as an educator, rather than as a researcher. So I’m not here talking as a researcher; I’m a researcher as an adjunct to the other things I do. One of the things I would say about openness is that it’s much easier to get people to agree to collaborate on projects with you if the outcomes are going to be open. People who are not usually perhaps accessible to you are fascinated by the idea, or interested that if they collaborate with you the outcomes will be widely available and they and other people can draw on them. So I think there’s an opportunity there if you are not part of a big research project but you are trying to get some research off the ground, you are more likely to find people to take part.
In terms of publication of research, what I can say is that I’ve recently been involved in editing a book, which is called Reusing Open Resources with Allison Littlejohn. And one of the things that I was firm about was that in order to have claims to be about open research we had to be open ourselves. So we managed to negotiate with Routledge that half the book would be published in an open access journal. I’m also involved in another book on learning design with Sandra Wills and James Dalziel. And that I know is also pursing the same sort of thing. So I think what I can find in terms of my own practice is that there’s an expectation that things should be open, and there’s an appeal in things being open. And I do make use of that, in terms of projects that I’ve been involved in.”
Martin Weller on where open has made a difference to his research practice
In lots of places really. So I was talking about something called Guerilla Research which is doing small scale research. So I think the idea there is that openness allows you to do different types of research. So it’s almost allowing you to rethink what it means to do research. So this idea of Guerilla Research, for want of a better phrase, is to do small stuff that you don’t need funding for, using open data, open tools, that you then blog about. So all that kind of stuff is suddenly made possible because of openness.
But in terms of what you might call more formal research… I’ve been a blogger for a long time; blogging is a strong part of my academic identity. That’s allowed me to create a global network of peers; which I think previously I would have had to spend all my time going to conferences, and doing keynotes, to keep that network up. And that’s led to very tangible results: so I’ve been invited to go and give keynotes in different places, and that’s because of my online identity, which you might regard as open scholarship.
But also it’s led to formal research projects, we’ve got funding for, through a network of people. And also just that I publish my stuff openly so it’s always open access. And if I don’t publish it in an open access journal, I’ll blog it and then tweet the results. So in many ways that kind of dissemination pattern has changed quite drastically, it’s not just about the article now it’s about the kind of conversation you have about it. And I occasionally write books which are published open access as well. And I’ve noticed that this kind of open approach all the way along makes a difference. So when I wrote my last book I went back and went through my own blog and found all the relevant stuff that I’d posted about it beforehand; and that not only gave me a starting point for the book – I wasn’t starting with a blank slate so it was good – but also I found that loads of people had commented on those posts which I could then follow up with those links, or which maybe got me to rethink what I’d been doing. So writing the book was a much more open process, so openness has really changed the way I work in that sense.”
Martin Weller on the disadvantages of open research
Yes, I think so. I think generally you shouldn’t see open research, open scholarship, in competition with traditional scholarship, as I think they can be complementary. But I suppose one obvious downside is that I don’t publish as much as I used to: traditional publishing. And I if that was really important it might be an issue. I think partly it’s because I blog and things; and in some ways that just scratches the creative writing itch, if you like, so I don’t need to publish, and sometimes it’s just a better way of getting stuff out there. Whereas before I might have written a conference paper, an article paper. But I think it’s a more efficient way of doing it but if publications are the one thing that’s important then it might impact on that.
I haven’t really had much of the issues that some people get … like getting involved with people trolling you, offensive behaviour. But then I work in educational technology, I know people that work in more controversial subjects, or subjects where there are big strong interest groups, like people who work on climate change for instance. If you put anything out in the open, you have to be prepared to defend it, spend a lot of time engaging with communities or just be prepared to take the flak. So I think it can be difficult. And I think occasionally it can be a bit of a time sink.
But I think in general I’ve found it… I think there’s an investment stage; when I started blogging, when I started using Twitter and stuff. It took time to do it, initial time. But then it reaches a payoff point where it actually saves you time. So now if I need to know something – I have a very good network – I just put a call out on Twitter, a lazy webcall, and it will save me time from doing it. And the same when I was writing that book, I think having access to that network saved me time. So I think there is an initial investment phase; but I generally it’s been positive for me and there’s not been many downsides.”
Transcripts for Open Research 4.1: Researchers Reflect on Reflection!
Leigh-Anne Perryman on the role of reflection in research
“So I think reflection has got an absolutely vital role in evaluating research projects, and all the contents of a research project, both large-scale projects where multiple researchers are involved and very small-scale projects conducted by a single researcher. In every case, people should be reflecting.
Thinking about what reflection achieves… in a way it allows for consideration of different ways in which the research activities could have been done, different ways in which it could have been planned, and from that it allows for identification of strengths – things that went really well – and perhaps things that didn’t go so well. The areas you could improve, in terms of research practices, processes, approaches. And things that could be done different next time, and if there’s not a next time for you, let’s say you’re reflecting on something at the end of your project, if you’re sharing that reflection, other people can learn from it. So, it’s a learning process, that’s absolutely vital. There’s not that much point in just reflecting and saying “OK, that’s done.” You’ve got to reflect and learn from it.
And … also in research it helps to identify your own assumptions, the assumptions you bring to the research, and any biases. We all come with baggage as human beings, we’re from particular backgrounds, we have particular assumptions and we live in particular mindsets in our perception of our world. And that inevitably will inform the way we conduct our research and the way we draw conclusions. Which is fine! It’s fine: if you reflect on it. So if you are very aware of what you bring to the research and how it’s affected your findings. And then explicitly share that, again not keeping it to yourself, but share it with other people, say “OK, this research is from a particular perspective, and this is my take on the research” and opening it up to other people then to put their perspective on it.
So I think reflection: vital to research. I cannot imagine any research project being rigorous and achieving valid findings and conclusions without reflection being involved. And so it’s the role of the responsible researcher.”
Tita Beaven on the role of reflection in research
“I think reflection plays a role in all stages of research. So that for instance when I’m setting up out on a research project, first of all I reflect on why I’m doing this, what it’s impact is going to be, whether it’s going to make a difference or not? I tend to do a lot of reflection, I suppose, at the design stage. How I’m going to carry out this research project. But I think it’s something that happens at all stages really.
So when I’m collecting data, I do a lot of qualitative research, so when I collect data from participants, at that same time I’m reflecting on the research questions, whether these need to be slightly tweaked and so on. And as I say I think it happens also at later stages of the research process so even when you’re analyzing the data, when you’re writing up, at all those times I think I do a lot of reflecting on what I’m doing and why I’m doing it, and what I want to come out of the research.
And I think it’s a process that doesn’t really finish when you finish writing up your research, because I think that then you also might present it at a conference, or you might discuss it with colleagues, and you get feedback as well. So I think you’re always constantly reflecting on what you are doing, as you do it and even afterwards. And I think that is also becoming more and more important for researchers these days; is to also really reflect on the impact that our research has more broadly.”
Tita Beaven on conducting research in the open
“Well I think it depends very much on how you work and whether you have colleagues or friends there to listen and act as sounding boards. I mean I’m very lucky that I work with colleagues who research in similar areas to me. So, yes, I think being able to discuss some of your reflections, I suppose, with others is a great opportunity. I don’t think everyone has it, and I think that some people are much more private when it comes to their research.
I think the other way in which we can be more open is also in discussing it, for instance, with the participants in our research. And I think this is something that we maybe don’t do quite as often as we should, just in terms of getting back to them at the end of the research and telling them what the outcomes of the research have been and getting their feedback on that. So, yes, I think you can be very open. I know that researchers who blog about their work in progress, there are researchers who present their work in progress at work in progress seminars and things like that. I don’t do that so much myself but … I think you can be quite open about it. And I think it’s something that should be encouraged actually, rather than seeing research as something that happens secretly and hiddenly and all you see is the end result of it when you see a published article. I think it’s actually quite useful to other researchers to see how the whole research process works and how sometimes how very painful it is! Or even things that don’t work or don’t go as planned. I think it’s very important to share not just the success stories but all the stories actually.”
Transcripts for Open Research 4.3: Reflection and Evaluation
Leigh-Anne Perryman on who should be involved in doing open reflection
“So when you are using reflection in the context of evaluating an open project, I think it’s very important for all stakeholders in that project to be involved in the process, to varying extents and in different ways. The core project team – that would be the researchers, the project manger(s) – they would need to be more intensively reflecting because that will allow for an ongoing process and learning from the results of that reflection. And that then needs to be central to the evaluation process. But other stakeholders, for example, funders, people upon whom the project is intended to have an impact, other collaborators, partners, they should in some way get involved in the reflection process. And that could be via things like feedback surveys, feedback questionnaires, interviews… anything that allows them to reflect, and you to learn from their reflection.
So with all these disparate stakeholders involved in the process of reflection it’s actually pretty important to have some sort of structure guiding that reflection, so providing a framework. So what you might do is look at the outcomes for your evaluation, the things you want to evaluate, the lessons you want to learn along the way. And then set up some sort of framework that prompts people to have a think about those areas, to reflect upon those topics. And then that allows for consistency across all of the stakeholders that are involved in the reflection process. So while it might seem a little odd to say: “Do structured reflection!” it’s just a bit of a prompt, a set of guidelines that makes for a more valuable reflection process.
Another consideration is that while it’s absolutely fine for some reflection to be totally personal and very private to yourself. And in fact when I was working as a Research Associate on OER Research Hub project there were in some times when I certainly wouldn’t be sharing the results of my reflection, let’s say if something went a bit wrong and I think “Right! I’m going to learn from that myself, but perhaps I better keep a bit quiet about it!” But generally it’s best if the results of reflection are shared and it’s a collaborative process, so this in turn will allow for a dialogue around what can be learnt from the reflection. In this instance it can useful then to have a reflection journal, a collaborative document, to which all stakeholders have access, or perhaps a core group of stakeholders have access, to allow all people involved to see each others reflection, have a think about it and then reflect on it.”
Leigh-Anne Perryman on when reflection should happen
“It’s tempting to say that you should be reflecting all of the time, especially when you are using reflection for evaluation. In fact we do reflect all of the time, life is a constant and ongoing process of reflecting on the actions that we do minute-by-minute, and thinking about whether they worked out and how we can do things differently.
But when we are using reflection as the basis for evaluating a project, as the basis for assessing our research and seeing whether we can make it more rigorous so that we can get more valid conclusions and findings then it can help to have a bit more structure in terms of when we reflect.
So roughly there are two times when one might reflect. There is reflection in action. So that’s reflection that happens during an event, during a decision point. And at this stage you may very quickly think “Ohh! This isn’t going so well, I’ll change and I’ll do something different.” So that’s pretty instant, pretty instantaneous, and in fact you probably wouldn’t find yourself recording that reflection at the time, the recording will come later. So that’s reflection in action.
The other type of reflection is reflection on action, which will happen after the event, the critical incident, the decision point. It’s where you sit down, give yourself some time for the reflection process, have a think about what worked well, what didn’t work so well, write it down, think it through and then plan for future action. And indeed planning for future action, reflecting on action, or for action, is another type of reflection. And really if you aren’t reflecting for action… no point in reflecting at all! So pretty continual but at key decision points as well.
So if at any point you’ve had to deliberate over a decision that you’ve made, have a reflect to think about why you made that decision, and the implications of it and then you can come back later and think about “did it all work well?”
Also this fits in with the need to record reflection. Because it makes sure the reflection gets done, if you’ve got a structure for recording that reflection. And it also allows other people to engage with the reflection and a constructive dialogue to be built up. Just thinking about some people who propose that reflection should happen right at the end of a project, to learn lessons for the future. In fact, it’s arguable that if you’re reflecting at the end of a project, it’s too late because you can’t learn the lessons yourself, the project can’t benefit from those lessons. A future project may, but why waste the opportunity not to learn from your own reflection, and let others learn, in the project you’re working on, at a particular time.”
