You Can’t Learn Indigenous Evaluation in the Classroom

Overview

The Interview

Gladys Rowe: Tansi, greetings! Welcome. I’m so grateful you are here. Dr. Melanie Nadeau aka Dr. Mel is an enrolled citizen of the Turtle Mountain Band of Chippewa Indians in Belcourt, North Dakota. She completed her master’s in public health in community health education with a concentration in health disparities and her PhD in social behavioral epidemiology at the University of Minnesota School of Public Health. Dr. Mel is a community-led scholar and has worked more than 18 years on various research and evaluation projects within the American Indian community. She has successfully engaged a multitude of tribal health stakeholders from across the nation and is dedicated to improving the health and wellbeing of Native communities. Dr. Mel currently serves as graduate program director and assistant professor for the Indigenous Health PhD program housed within the Department of Indigenous Health at the University of North Dakota School of Medicine and Health Sciences. Dr. Mel also serves on the Turtle Mountain Band of Chippewa Indians Research Review Board; the American Public Health Association’s American Indian/Alaska Native/Native Hawaiian Board; and as program co-chair for the American Evaluation Association, Indigenous Peoples in Evaluation Topical Interest Group.

Welcome, Dr. Mel! I’m so excited to share the space with you today and to jump both feet into this conversation we’ve been planning. Before we get into some of the questions that I have prepared, I wanted to see if there is an introduction to yourself that you wanted to share into this space?

Melanie Nadeau (Dr. Mel): Yes, thank you so much for having me. I’m excited to spend some time with you today. Melanie Nadeau indizhinikaaz. Zhaawani Ghizigo Ikwe indigoo. Makwa indoodem. Mikanaak Wajiw indoonjibaa. Alvarado, Minnesota indaa. So, my name is Melanie Nadeau. My spirit is known as Southern Sky Woman. I’m Bear Clan and Turtle Mountain is where I come from (Turtle Mountain Band of Chippewa Indians). I live in Alvarado, Minnesota.

Gladys: Thank you so much for bringing yourself in in that way. In the introduction I shared, there are so many different roles that you hold and focuses that you have. I know this has been a journey that you’ve taken over a couple of decades, so I wanted to hear a little bit from you: What has this journey into Indigenous evaluation looked like for you? Where did it begin and how have you gotten to where you are right now?

Dr. Mel: Yeah, so I was very fortunate when I went to the University of Minnesota School of Public Health to get my master’s degree. I ended up getting a graduate research assistantship right away with the Children, Youth and Family Consortium. And as soon as I got there, I was tasked with working with stakeholders that were involved in trying to improve parent involvement with their children’s education. I ended up doing my master’s thesis with that group and conducting an evaluation of that particular collaboration, and I really explored the intersection between health and education disparities and how that impacts the education of these youth in the school systems. It was very eye-opening. And in my master’s program I did take an evaluation class and I was working on my master’s in public health and community health education with a concentration in health disparities and really getting trained to be a research methodologist at the core of that.

And when I first started my journey, I was trying to figure out, How does evaluation differ from research? That was one of the things that intrigued me about evaluation. And I realized very early on that it was a different process and a different way of going about doing the work. The next semester I ended up getting another graduate research assistant position with the American Indian Community Tobacco Projects in the Twin Cities. I conducted the evaluation for one of their initiatives. There were a couple of initiatives underneath that: first, the American Indian Community Tobacco Survey, where I really started learning how to write surveys, how to make sure that they were culturally appropriate when looking at the harmful effects of commercial tobacco. The second initiative underneath those tobacco projects was about honoring the medicine, the traditional tobacco for the people. In Circles of Tobacco Wisdom, there were these various cohorts of Elders that would come together monthly, and they would have talking circles and prayer circles. They’d open up with smudge. There was an Elder there that would lead the talking circle and pass around the feather, and the Elders present would take turns talking in the circle. No one was above the other, and everybody was equal. And I was tasked with the evaluation of these gatherings with these Elders in four different cohorts. How do we write that into an evaluation? Because there was so much good about the traditional medicine, all good actually. So, I started working with state stakeholders to ground the work culturally so that it was digestible to the community and it was important to them and it was coming from their perspective. It was just really, really sacred work. And I feel so very blessed that I was able to be involved in that work in the Twin Cities where you can feel very disconnected in an urban setting. Even though I was seven hours away from my reservation, my community, the Turtle Mountain Band of Chippewa Indians, I felt very connected to community because I was able to work on those projects. And I don’t think that I would’ve done as well if I hadn’t been involved in evaluation from the onset.

Gladys: Those are some wonderful examples and thank you for sharing the experiences that initially sparked your interest and really moved you forward from your master’s program. I love the example you shared of the four cohorts of Elders. I was wondering if you could share: What were some of the mechanisms that you used when you were gathering the information to evaluate that work?

Dr. Mel: It’s so interesting that you asked that because, <laughs> if I’m going to be honest about it, I didn’t know what I was doing, and they said, Hey, you’re going to be in charge of evaluating this. I was a graduate research assistant. I didn’t have experience really doing that. I had taken a class, but it didn’t really grab me at my core. And I think, looking back at it now, that’s because evaluation wasn’t presented in a way that made sense to me. And now that I’ve been practicing this work for 19 years, I look back and I realize that it didn’t grab hold of me because evaluation is very values-based. If you’re working within somebody else’s framework, it doesn’t translate well to the work that you do. Fortunately, I was working with Native people in the Circles of Tobacco Wisdom, with these Elders.

So, I was given a space to be creative, to explore, to be innovative. And they trusted me. They’re like, We trust you, Mel, and so I followed my heart. I followed my heart in the work that I did, and I just operated and functioned in a way that felt good and felt right. So I just stayed true to my intuition. And when I was working with the Elders, I would gather the information and go home and summarize it and I’d go back to them and I’d say, Hey, did I get this right? And we’d go over it together and they would say, No, that’s not really what we meant. This is what we meant. And they would teach me how to translate their information as a collective. And then I would make those adjustments and then I’d give them a copy.

And the one Elder had lived in the city since he was younger, he was an individual who had been relocated out to the Twin Cities. He said, Mel, I have been involved in a lot of initiatives and you’re the first person that has ever given back the results. And I stood in that moment, and it just struck me: that’s not in alignment with our values. There’s this underlying piece that if you ask somebody for something then you should be willing to give. And you may be asked to serve in a different capacity than the one you know you’re doing, because you’re viewed as a resource. Your gifts are honoured, the people that you work with might ask you to do X, Y, or Z, and it might not be in direct alignment with the initiative that you’re doing. And I think that’s a piece of the work that is unique, but also that I really appreciate. And so that really stuck with me and I thought, The first people that I’m always going to think about, whether I’m doing research or evaluation, are the people that this information’s coming from. And they’re always going to be a priority and they’re always going to take the front seat when it comes to getting back the results.

You know, working with the Elders, I let them define things. So, we had money in our grant for t-shirts, and they bought t-shirts, and they drew the turtle on the t-shirts. They made a cake with their turtle. They just loved their turtle and they put that turtle on everything! They had journals, they had the turtle on their journals. But the t-shirts were to honor people in the community. And they said, Who can we give the t-shirts to? And I said, Anybody you want, anybody that you would like to honor. Well, they were so used to being told how to operate within their projects that they didn’t feel safe and comfortable making that decision. And I was like, No, you can actually give them to whoever you want. You get to define what success is. What does that mean to you? You get to define that. And so one Elder said, Well, you know what, my niece, she tried to quit smoking. She didn’t make it very far, but she tried. And that’s the important thing, that she tried to quit, and so I’m going to honor her with a t-shirt. And I said, That’s great, that’s wonderful. “And they said, Who can we give the t-shirts to? And I said, Anybody you want, anybody that you would like to honor. Well, they were so used to being told how to operate within their projects that they didn’t feel safe and comfortable making that decision. And I was like, No, you can actually give them to whoever you want. You get to define what success is.”
_{– Dr. Mel}

The other thing that was unique in working with the Elders is that they made their own brochures. Each cohort created a different brochure, and we would present them. You know, I have always looked at myself like a vessel. The community leads everything that I do, even for my research. And I love being a methodologist because I bring my toolbox and I say, Here’s some ideas. What do you think? And, you know, if they want me to make the decision, if they ask me to, then I will. Otherwise, I give them the space to do that. And the brochures all came out different and they were all so beautiful. And two of the cohort members brought in their grandson or granddaughter to create art. They honored their grandchildren and put their art on their brochures.

A lot of really neat things happened with that project. They learned how to grow traditional tobacco and they all started seeds. And it was really interesting: at the end, we had worked our way all the way up to the policy level, policy considerations. And once they found out how the tobacco industry manipulated them, manipulated our people, the Elders were really offended. And we actually had a few Elders that quit smoking even though they had smoked all of their life. Just knowing that they were being targeted and manipulated by the tobacco industry totally changed their perception. And they started going to the capital and being really involved at a political level. So, we started at this individual level and then we ended up at this political level. And also, there were people that would go and work in the schools and teach at the community level. And so they stand for the whole socio-ecological model in the presentation of their work. But I would say that it was very sacred work because it was done in a traditional way. Some of the Elders in the cities are from the same nation, but many come from different nations, so it was a very diverse group. That’s what I try to explain to people. I say, Native people, you know they’re the smallest population in this country, but we also represent the largest diversity. And I had firsthand experience with that because we’d have traditional speakers come in for the different nations to share cultural teachings. And that information was translated with regard to tobacco in different ways for each nation.

Gladys: Thank you for sharing those examples. It resonated quite a lot for me in terms of how you shared in your master’s education: you took one evaluation course and then you got out there as a graduate research assistant and realized <laughs> that you needed to actually start from a different place. In the work I do, I talk about my evaluation journey, my evaluation learning. I continue to learn a lot more from community than I did from the evaluation courses that I took back in my master’s program. There’s a lot to say about how that needs to change, maybe, but also there’s a lot to value in our ways within our communities. We can continue to learn about the implementation of our values in evaluation. I’d love to hear where you went from there in terms of your evaluation journey. And you said you have this realization, looking back, that the work you do is really values-based. I wonder if you could talk a little bit more about that and where you went next.

Dr. Mel: So, I went on for my doctorate in social behavioral epidemiology. While I was doing that, I worked for the community tobacco projects, I had the honor of working with these Elders for about five years. And then I got recruited out of graduate school to launch the American Indian Public Health Resource Center at North Dakota State University. And that was interesting because it’s like, Okay, you’re going to start August 8th, and your team of four public health professionals will be starting the same day. We took a four-pronged approach to public health by looking at policy, research, services, and education. I had a master’s level public health professional in each of those arenas. Then I was the operational director tasked with launching the center that would provide technical assistance to tribes first in the state of North Dakota, then in the region (that was our five-year goal), and then in the nation (our 10-year goal). Well, we had our national reach within one year. <laughs> After we got the center going, we started traveling a lot to provide technical assistance to communities. And then all of a sudden an opportunity came up and we were doing evaluation here and there on some meaningful projects.

The one that I want to touch on is our work with the Minnesota Department of Health. They put out a call for an entity that could work with Minnesota tribes on the Statewide Health Improvement Plan projects to implement an Indigenous Evaluation Framework.^[1] And we were like, Oh my gosh, what? It was the first time that they had ever allowed the space for that. I’ll have a publication coming out in Roots and Relations this summer that goes in depth how our center from a process standpoint did the work and applied a framework. Because frameworks are nice, but they don’t really translate to the ground. How do you translate that framework into actually doing the work? And so we decided that what we were going to do was we were going to work with the communities, meet them where they were at, and build capacity around evaluation. Because evaluation, much like research, had a bad name and we needed to work to destigmatize that process. And there had been projects done before where the communities’ response was, Oh great, we gotta do evaluation. Ugh. You know? And the way that it was being done was so – I would say that it was very limiting. And I feel very comfortable working on the frontline. I feel very comfortable not knowing the answers. And I feel very comfortable being innovative and allowing space for creativity.

And I feel like in the evaluation space a lot of professionals struggle with that. And so I stood strong in my belief that this work had to be done in a different way. So, we created a number of activities, and we went out and talked about the CDC [Centers for Disease Control] model of evaluation^[2][3] and then we rolled into the Indigenous Evaluation Framework. One of the things that I noticed early on when we started creating a process that would make sense from an Indigenous perspective was the difference between that and the CDC model. The CDC model will say, You must be willing to report the good just as much as the bad. And that does not resonate with tribal communities. It just puts things in a different space – it doesn’t go so well. The Indigenous Evaluation Framework says all knowledge is good knowledge, and even if you learned something that you didn’t anticipate, that’s a win. That’s innovation and that’s something that we can work with. That really resonated with the tribes. We worked with 10 of the 11 tribes in Minnesota and that stood out to them.

And then we went into this activity where they had to introduce themselves. We did an icebreaker and we said, Okay, you’re going to draw out who you are. You don’t have to be an artist, just draw whatever’s important to you and then we’re going to scramble the cards in the front of the room and then call you up and you pick a card, but you can’t read your own card! And then someone would come up and pick a 3 x 5” card and say, Oh, okay, this person likes to fish and hunt, looks like they have two dogs. Sometimes they’d get it right, sometimes they’d say something funny. And it was just a lighthearted way of everybody getting to know a different side of each other. And these people already knew each other because they worked together at the program level. Of course they didn’t know us. There was a mix of people who knew each other and people who were just getting to know each other. But everybody touched on their animals, their children, their culture, the things that were most important to them at their core, what they valued.

So, in those introductions, everybody got to know each other in a way that was meaningful to them. And then we went on and we’d have training and then we’d have another activity, like an activity where they would identify all the stakeholders for their project. And I mean they identified stakeholders at the community level, the institutional level, the state level. They had all these stakeholders that they were connected to. And then we also had another activity that we did that was cultural grounding. And we said, Okay, now we’re going to do a values activity. We want you to think about your values in relationship to your programming. And so they’d write out their values and love was always present, it was always there. Love, trust, bravery.

And they would write out all of these values that were important to them, and then we’d say, Okay, now define that within the context of your work. So, they would go through and say, How do I describe this word within the context of my work? They wrote it all out and it got them grounded in their theoretical framework, their worldview. And so they got to start from a place that was important to them. We facilitated this conversation, and we had all these different sticky notes literally covering the whole wall.

I guess that’s another thing that was different with our evaluation is that we did it as a team and the communities really liked that. So, it wasn’t just one person coming out saying, Well, I’m here to evaluate, give me your information and I’ll come up with something. It was really a process right from the beginning. And so we did this reverse logic modeling process with them. We didn’t start right from the beginning of the project. They’d had two years with mainstream evaluation, but it wasn’t working out. So that’s why they decided to hire someone to do an Indigenous evaluation. And so they already had a couple years behind them on their projects and we got to come in and reset them within an Indigenous Evaluation Framework and we said, Okay, let’s talk about your long-term goal.

We worked with them to develop a statement that was relevant for where they were at in their programming. And so, they came up with their long-term goal and then we worked back from that to intermediate goals, short term goals, all that fun stuff. And then we’d do a lot of shifting depending on the context of everything that was happening, the big picture. And then we fleshed all this out and they were just blown away. They’re like, Wow, we didn’t realize we were doing all of this work! And we didn’t realize that all of everything that we do is connected to this long-term goal! And that’s when I realized that our communities really could benefit from evaluation. I always poke fun and say, Everything’s connected, Native people know that. We know that everything’s connected. So, when it comes to framing and doing a concept model, that should be no problem for us: we know everything’s connected and that everything affects everything else. That’s something that’s a strength for us. I think that we can build our capacity in that way as Native people around programming to get our initiatives funded in a way that makes sense for us. And so, we fleshed out all of their work and then we compared it to the state work plans and we realized that over 50% of the activities that these programs were doing were not being reported on the state tool because the tool was from the state perspective, and it wasn’t allowing space. It was actually stifling their reporting process and it wasn’t allowing the space from a cultural-based view to share their successes.

Gladys: Wow, what a journey! It sounds like it was quite an experience for you and your team to be able to support communities in that way. One of the things that you touched on there was why Indigenous evaluation is important, and I wondered if I could just probe a little bit to ask you to make some of those connections explicit. For you, why is Indigenous evaluation important for us and our communities?

Dr. Mel: I always say, Hey, I’m Native first, I’m all these other things second. <laughs> At my core I’m Native first and that allows me the space to be who I am in community, and I really love that. So for me as a community member and methodologist, I love the fact that I’m doing some pretty innovative things with evaluation methods. I’ve just grown so much over the years and it’s kind of fun to see where I’m taking it. I always like to say I’m breathing life into it. I’m breathing life into this process and letting it go where it’s meant to go. The other thing that I really appreciate about evaluation is that it allows space for our artists. You see it come out a little bit in the people that translate that at the community level. They are our Knowledge Keepers, and their art holds teachings and stories. And I would even go as far as to say that everybody has some artistic vibe in them and the artistry at the community level is our worldview and our theoretical framework from that perspective. And so to me our evaluation is an underutilized tool. It’s misunderstood and I feel like it’s an opportunity where we can really build capacity in a way that makes sense for our communities, and we can actually advance the health and wellbeing of our communities using this tool.

When we were working with the Minnesota tribes, all of them would say, How do we measure community-wide impact? We want to look at things more broadly – we don’t just want to look at Joe smoking or, you know, Joe’s diabetes. They were working on diabetes, nutrition, education and then tobacco projects. But they said, We don’t want to look at these individuals, we want to look at the collective. We want to see how this is impacting our community as a whole. And as a methodologist and a researcher, I replied, You can do that. You can actually apply a research methodology to your evaluation implementation and measure community-wide impact. And that is something that they wanted to do. And so, in one of my recent publications, I said that the more that we can identify opportunities to build protective factor-rich environments and invest in that, the better. Yes, investing at the community-wide level is expensive. But I would say that we are dealing with the largest health disparities in the United States and we are the smallest community. I think it’s good value for the money and it would be a good investment to invest in that approach.“…the more that we can identify opportunities to build protective factor-rich environments and invest in that, the better. Yes, investing at the community-wide level is expensive. But I would say that we are dealing with the largest health disparities in the United States and we are the smallest community. I think it’s good value for the money and it would be a good investment to invest in that approach.”
_{– Dr. Mel}

Gladys: Absolutely. And that question around impact I think is widespread right now. You know, wanting to think about the collective impact of investing in these kinds of opportunities. Like you said, What can investing in the protective factors mean for the next generation, for the generation after that, for seven generations forward?

I was wondering if you could talk a little bit more about evaluating for impact. What are some of the things you might be thinking about or questions you might have? Or what might have you learned in your time evaluating that?

Dr. Mel: Well, a couple of things. As far as the impact goes, this is more my research realm, but it’s still relevant, because I’m joining these processes to advance health and wellbeing. I just did a mixed methods literature review looking at impact, because we know that sexual health indicators, diet, physical activity, nutrition-related indicators lead to cancer over the lifespan of individuals. We also know that substance use-related behavior leads to cancer over the lifespan. What we don’t know is the risk and protective factors for those cancer-related indicators for our youth. And that’s the main research that I work on. I’m currently working with my community to build capacity to do this work.

And I have an individual that works for me, a doctor that’s in Turtle Mountain and he’s currently scheduling key informant interviews and focus groups. And I work with the community to figure out the process, like how they wanted the focus groups done, who they wanted interviewed. We explore the question, What are these risk and protective factors that are present with Turtle Mountain? And eventually we’re going to create this tool. We’re going to validate this tool and then we’re going to administer this tool to the youth. At one time, my younger self never would’ve thought I’d get to do this. I’ve always wanted to work with youth. In fact, my elevator statement before I went on to get my master’s degree was, I want to create culturally grounded interventions and hopes of closing the health disparities gap with my people in Turtle Mountain, and that is still true to this day. My goal hasn’t changed.

What I realized though when I went to the university is that there’s no data. And then the data that is collected is so poorly collected that it doesn’t tell you anything. The use of national data is abused. I say unless you’re doing a count there’s no reason to use national data to predict things. We need to advocate for meaningful data collection within our communities because we haven’t been given that privilege, and data translates into opportunities for our community. And so then I realized that our communities weren’t involved in the tool creation process. So I’m like, Okay, I’ll work with them on that and then I have to collect data. I’m hoping that a few years down the road I’ll have the data that I need and then I can go back to the community and say, Okay, this is what showed up. This is the risk and protective factor profile for substance use. What do you want to focus on for intervention?“We need to advocate for meaningful data collection within our communities because we haven’t been given that privilege, and data translates into opportunities for our community.”
_{– Dr. Mel}

My community makes all the decisions. I had this plan going in saying, I’m going to do focus groups with the youth, and I’m going to do key informant stakeholder interviews with adults who work with youth. But, I said, they’re going to have the final say. So I brought them my ideas and they’re like, Well, no, we want you to approach it this way, that way. They adjusted my questions and then they said, We want you to actually interview youth and we want you to interview adults, but we want you to have focus groups with behavioral health specialists. We want you to have focus groups with principals and other parts of the school systems, administration. We want you to have a focus group with the teachers and then we want you to have a focus group with the cultural knowledge bearers. You know, they felt like the touchy subject would not be good for youth in focus groups together, and that youth would need to have their private space to discuss those things. And so they totally flipped my design right on its head. <laughs> And I welcomed that. I was so happy!

So, coming back to impact, I think that evaluation can definitely measure impact at various levels. The point that I wanted to make is that there’s so much in the literature that we can use to our advantage to say, Hey, we know Y leads to Z. We don’t know what the relationship of X to Y is, but we know what Y to Z is. And we can use that in framing our initiatives. When I was younger, I would say, Oh geez, there’s no way that I could find out what the risk and protective factor profile is for cancer over the lifespan of youth! Well, today I know I can, because I know how to frame existing data and I know what significant findings and salient findings have been found on the aggregate. And I can use that to translate for the intermediate or risk and protective factors that are present. Because we know that they lead to these long-term health impacts over the lifespan.

Gladys: Thank you so much for drawing that out for me and for the audience. And you shared that you’re a researcher, you’re an evaluator, you hold all of these roles, but they’re all interconnected; of course they’re interconnected! And I was reflecting on the pathway or iterative cycle that happens from research to intervention, to funding, to evaluation, to results, this cycle that happens around knowledge production, knowledge generation. It all really – when you think about Indigenous evaluation and Indigenous research – it matters who is doing this work. And I know this is said in so many different spaces, but I wanted to bring it into this conversation because the examples that you’ve been giving illustrate that who is doing this work matters. It matters at the research level, it matters at intervention design, it matters at tool development, it matters at evaluation, it matters in funding spaces. Who is doing this work matters because the way that we see the world and tell the stories of the world is different. And it goes back to the connection that you made to listening to your gut at the beginning when you were a graduate research assistant working in those community circles. What does our gut tell us? Our gut is connected to our worldview and our values! I sound really excited because I am excited. You have this thread that has gone through all of the conversation that I want to honor. So thank you for drawing that out.

Dr. Mel: Yes. There’re two other things that I’d like to touch on specifically. When it comes to the translation of the information out to community, that step is so very important as well. And I’m working with an individual – you know that I mentor other Native scholars on how to publish their dissertation work – and in this process, I said to this individual, I evaluated your data, and we had this question. And I’m like, Okay, how did you frame it? Because they had this cultural immersive experience for professionals, a really, really, good cultural immersion, a retreat for non-Native professionals so that they could be more aware in their clinical practice. And so I said, Well, that’s good, but I was looking at the data and the findings, and I’m thinking, This doesn’t feel like a cultural retreat reading the findings, you know?

And I said, Who did this? And she’s like, Me and a couple others. And I’m like, Okay, how did you decide on your translational framework that you would use? And she started spouting basically the academy view for the school that she was in. Well, that’s typical, right? But you know, it didn’t feel right to me because this was a cultural immersive event. And I said, Okay, you worked with an Elder. What did the Elder think when you talked to her? Did you ever talk to her about how this was going to be framed or translated? You know, when you wrote up the results? And she said, Yeah, actually I did. Well, what did the Elder tell you? She said she wanted me to frame it within the seven teachings. Oh, I said, Wow, that’s really cool. Hey, why don’t we pull your data and reframe it in a cultural perspective, in a cultural framework? <laughs> So now you’re moving into the space of reframing the translations of findings from a cultural view that makes sense from the community perspective. And then she’s so excited, she’s like, Oh my gosh, I’m so excited for this. And I said, Okay, these values, you need to define them within the context of your work. So now it’s kind of the backend process, right? You know, normally you do that in the beginning, but then if you’re going to do that for results, then you do that on the backside and you frame it within that to translate it back out so that it makes sense within the context of the work that’s being done.

Another thing that’s really been fun is I teach Indigenous Evaluation Frameworks. And I have students that come in and I teach them how to relate tribal processes, tribal teachings with evaluation processes. And we ran a data science academy: Can you align tribal process with the data science cycle? I have this student that’s really good at it. I’m like, You’re going to be a theorist. I already see it. She said, Okay, this community had the horse and travois, and she broke it down: this is built this way, and this is the point of this. And then she aligned it with a data science process. She’s able to translate that information. And then she did another breakdown using the lodge and the role of genders within the lodge, because our gender constructions are different as well. She just does this amazing work.

I was asked by the principal investigator at Mountain Plains Mental Health Technology Transfer Center at the University of North Dakota – they provide training and technical assistance to workforce in the region – to facilitate some conversations with some tribes. And I said, Yeah, I’d love to do that! You know, it’s been a couple of years with Covid. I want to get out of the house and be around people again! Absolutely, count me in. But I have one condition. What is that? they said. I said, You’re going to have to dump whatever idea you have and let me lead. You’re going to have to trust me and you’re going to have to trust the process. And they said, Okay. So we went and facilitated some conversations. And I had to pull them in a couple times and say, Hey, remember I told you to trust the process. Anyway, my student and I went in there and we just worked together to facilitate these conversations. The first day we had them break out into groups and then they all made these stickies, and they were everywhere. And then we reframed the stickies within the socio-ecological model to see what level things were falling at. And then we translated all this information back into a community report. And then the university technical assistance providers pulled me in, and they said, What about our report? Because we want to write a white paper. I said, Write a white paper, then. <laughs> I said, I’m doing the community report. Remember, that’s what I came in to do, and that’s my first priority when I do this work. And that’s where everything goes is into the community report. And people are just not used to that.

But now that the university technical assistance providers are seeing what’s coming out of it, they’re more interested in supporting the community report. For instance, the community report arose organically out of the first community discussions. First, my student and I managed to get about 60 individuals to introduce themselves in 60 minutes, believe it or not. How did we do that? The 3 x 5” card: Write who you are, draw who you are. We gave them five minutes to draw who they were. And then they stood up and described their card and they were able to do that within a minute. While we went through those, my student was sitting there creating a word cloud. And the biggest thing in that word cloud, the word that rose to the top was family. Then, all these other factors, you know: animals, being together, healing, the mountain, the environment, the land, everything that we already know that’s at our core as Native people. Well, then she put that word cloud into the community report. She put it within the shape of a buffalo, she started the report out with a traditional teaching that was grounded in that session. It was just amazing. And then in another part of that session, she used graphic imagery, strategic illustration I think is another name for it. She used all of these little images and created this mountain with these people. And she described the cycle of going up the mountain on the west side, going up the mountain and sliding back down and being stuck in this western model of healing that just doesn’t quite get us over the mountain. Some people get over the mountain, but they don’t arrive down with all the people. And she’s like, We’re not going to heal until we’re all together. She created a storyboard, and she shared that storyboard and she said, Hey, did I hear you right? This is what healing sounds like. This is how we go from trauma to resilience. Did I hear you right? And everybody there was blown away. And now we’re hoping to present at NIHB [National Indian Health Board Conference] in Anchorage, Alaska – or rather the community partners and my student hope to present. So, yeah, I have a lot of fun with the work I do, bringing these methods that are rooted in Indigenous evaluation and actually applying them across the continuum. And it’s been very effective.

Gladys: I love that you shared that you’re teaching Indigenous Evaluation Frameworks within the program. What an opportunity for building capacity for Indigenous evaluators, which is needed so badly, that capacity for us to do this work for our communities alongside our communities!

When you think about the field of Indigenous evaluation and new evaluators, emerging evaluators, what else is needed to support or strengthen this field based on your experiences and observations?

Dr. Mel: I think that the more opportunities that we can have to work in partnership with community, the better, because you can’t learn Indigenous evaluation in the classroom. You can hear examples, but you don’t realize until you’re out there doing these processes and seeing how effective they are. My student, for instance, is like, Oh, I love evaluation! <laughs> And she wants to keep going down that road. And on this journey together, we need a framing of health and wellbeing in the community in a way that makes sense for the community. I feel so strongly about Indigenous evaluation, that we really have an opportunity here to advance the health and wellbeing of our people as a collective. And in that we can have more opportunities to do this work in a way that honors the process, because it’s heartfelt work. If you’re going to do good work, you have to allow space and time for this process to unfold. You know, you’re not going to have your report in a week. It takes time and thoughtfulness and reflection, and you have to give that space to this process in order to do the work in a good way.

Gladys: So, so, so true. I was wondering if you had – I’m sure you have lots of insights and wisdom that you share with your students – but I’m wondering what you’d like to share specifically with the audience of Indigenous evaluators who listen to this podcast.

Dr. Mel: So, one of the things that I’ve been working on, and it’s coming … I’m just trying to figure out how to translate this process within the classroom. This fall will be the third time that I’m teaching the class. Their final project is to create an Indigenous evaluation toolkit. And you can imagine 48 students in our program, and I have maybe 10 to 15 students at a time for each cohort, students representing people from across Turtle Island. So, we are developing a toolkit that brings up these considerations that apply to us all. And that collective is really what I want to come out of this Indigenous evaluation class.

I have this opportunity, but never in my lifetime would I have thought that I’d be sitting at a university! You know, I co-launched the Indigenous Health PhD program with Dr. Donald Warne, and now he’s moved on to Hopkins and I’m sitting here with this program, and I never thought I’d be in this space, in this place of teaching mostly Native doctoral students. And I feel so very privileged and honored to be in this space. And so I really want to tap into that collective knowledge that they bring to the Indigenous health PhD program. One of the criteria for admission is you have to have lived experience working within Native communities. You don’t have to be Indigenous to be part of our program, but if you can’t show that you’ve worked with Native communities, you can’t get in.

So 45 of our students out of the 48 are Indigenous to Turtle Island and they are working professionals. And it’s so powerful. I really can’t describe how powerful it is to be a part of this group. And we have our students saying that they’re healing. I mean, how many students have you heard saying that they’re healing by going through their PhD journey? <laughs> But when we do things in a good way and we do things in a way that makes sense to us, it is very healing and it’s very gratifying. And it’s the first time that these students have been in an academic space and felt validated at their core that they were welcome in the way that made sense to them. We’ve made space for our students to start our classes with prayer or in any way that makes sense: some of them do meditation, they’ll do some stretching, mindfulness, share a poem, they’ll share a fun fact or something that made them smile this week. And so they have all of these resiliency tactics that they use to ground themselves in a good place before they get to the work. And so I just feel very fortunate and really want to tap into the collective knowledge, because I’m learning being in their presence just as much, if not more, than they’re learning being in my presence. And I want to bring that collective knowledge to Indigenous people across Turtle Island in the form of an Indigenous evaluation toolkit that taps broadly into those considerations, around ethical considerations, around the importance of positionality, around knowing where you come from and knowing how that impacts the process of the people that you’re working with and owning that. You know, the future looks bright for sure.

Gladys: It certainly does look bright. What an inspirational space! I have to say as someone who just finished a PhD a couple years ago, to hear a student say that they’re feeling like they’re healing in that journey – wow. I cannot wait to read and to learn from this amazing collective of PhD students in your program.

Dr. Mel: Yeah. They’re really amazing. And of course they have their moments, you know, every PhD student does. It’s definitely tough. But yeah, it’s just been a beautiful experience maintaining that rigor and also trying to allow that space where they can be creative and grow as Indigenous health scholars so that they can do the good work when they get out into the community.

Gladys: Exactly. We are getting to the end of our time together. It’s flown by and I have certainly appreciated all of the stories that you have brought into this conversation. I wanted to offer an opportunity for any last thoughts or words that you wanted to make sure to share before we close off our time together.

Dr. Mel: Thank you, Gladys, for doing the beautiful work and allowing space to have these conversations. You know, I’ve listened to the podcast episodes and all the beautiful teachers that you have brought in. As it’s women’s history month this month, I really want to honor the matriarchs that shared their stories. I can feel their collective voice and relate to all of the teachings that they have shared. This is such important work that you’re doing, and by doing this, you’re allowing me to breathe life into the process of evaluation at another level. So I thank you for that. Miigwetch.

Gladys: Ekosani, I thank you so much as well for the generosity of the sharing that you have brought today, and I look forward to continuing to learn from you. So wishing you a lovely rest of your day.

Dr. Mel: Thank you.

Gladys: Listeners, I’m so glad you spent time with us today, too. I look forward to sharing the space with you again soon. Ekosani.

Footnotes

About the author

name: Melanie Nadeau, PhD, MPH (Turtle Mountain Band of Chippewa Indians)

institution: University of North Dakota

website: https://campus.und.edu/directory/melanie.nadeau

Dr. Melanie Nadeau, aka Dr. Mel, is an enrolled citizen of the Turtle Mountain Band of Chippewa Indians in Belcourt, North Dakota.  She completed both her Master of Public Health in community health education with a concentration in health disparities and her PhD in social/behavioral epidemiology at the University of Minnesota School of Public Health. Dr. Mel is a community-led scholar and has worked for 20 years on various research and evaluation projects within the American Indian community. She has successfully engaged a multitude of tribal health stakeholders from across the nation and is dedicated to improving the health and well-being of Native communities.  Dr. Mel currently serves on the Turtle Mountain Band of Chippewa Indians Research Review Board, American Evaluation Association Indigenous Peoples in Evaluation Topical Interest Group, and American Public Health Association American Indian/Alaska Native/Native Hawaiian board.  Dr. Mel also serves as Vice Chair and Associate Professor for the Department of Indigenous Health at the University of North Dakota School of Medicine & Health Sciences.