As Costanza-Chock (2020) points out in the concluding chapter of Design Justice, “we urgently need more critical analysis in every design domain.” What critical analysis might look like for instructional design, however, is complicated for practitioners who are rarely afforded final authority or power over the design itself. While existing scholarship in this area offers a variety of approaches and considerations, the literature does not point to a single ideal framework or list of steps for assessing instructional design in terms of inclusivity, power, equity, or justice. In order to get closer to critical practice of instructional design, we in the field must blaze the trail ourselves. In this chapter, I explore the ways power, process, and positionality influence course design from a perspective many instructional design professionals are familiar with: an inherited course, that is, one we did not design ourselves but are now tasked with maintaining and improving.

For practitioner-researchers like myself, this leaves room for exploration into how one might critically analyze an existing instructional design in an effort to develop a framework for designing more just learning experiences from the start of the process. The more we practice these types of design approaches, the more mature the processes will become (Costanza-Chock, 2020). To begin, I decided to analyze a course in my instructional design portfolio based on existing critical scholarship in order to practice interrogating these concepts as they play out in a course design.

The following work is an effort to create salient applications of themes identified through research and practice regarding design justice in instructional design contexts. In a 2020 article, Collier (2020) draws on the writings of various educators which implores us to approach critical analysis of designs by working to uncover “what’s wrong” in order to take action towards counteracting marginalizing designs in education. This paper draws on critical scholarship in the learning sciences to guide the analysis and help other instructional designers focus their own critical analyses of existing course designs by providing examples of my thinking about these ideas in relation to specific elements of an online course. I also attempt to synthesize my process into a framework as a starting point for improvement with future critical analysis cycles. At the conclusion of this paper, I summarize my discoveries and reflect on the process and my role as an instructional designer.

Framing Concepts

Instructional designers are well positioned to influence the design of learning environments, and we should be “accountable for the social and political consequences” of our work (Barab et al., 2007, p. 296). I view my role in this critical analysis as similar to that of Barab et al. (2007). I aim to “position [myself] in a manner that will attune [me] to extant issues and highlight them among the community” (Barab et al., 2007, p. 281). The community for this work includes not only instructional designers on my team and instructors who teach this course, but anyone interested in critically analyzing existing instructional designs and how their discoveries are connected to, and could be changed by, the processes they follow to design instruction.

To begin the analysis, a review of critical learning sciences research was conducted to derive a framework to evaluate the instructional design of the focal course in this analysis. “Questions of power and ideology come to center stage in the decisions that educational designers make” (Barab et al., 2007, p. 291), and it is important that this framework is a flexible one that could potentially be utilized to conduct a similar analysis of other courses. This effort attempts to critically interrogate an existing design with the goal of uncovering areas for improvement, or, as Collier (2020) writes, to discover “what’s wrong” with a design.

Taylor (2018) outlines some commitments to ethical teaching and research, such as the commitment to “foregrounding issues of historicity, race, power, and privilege in the curriculum I teach and/or design” (Gutiérrez & Vossoughi, 2010, in Taylor, 2018, p. 196). Taylor identifies some questions that guide this commitment, which I find particularly valuable to this work: “What voices are unaccounted for here? Why might that be? How can I/we fix that?” (Taylor, 2018, p. 196). While these questions of representation won’t reveal every opportunity for improvement, beginning a critical analysis with this mindset will uncover details about the course design that can lead to other avenues of critique.

Barab et al. highlight power and its role in design. Indeed, power is conveyed, wielded, and distributed in different ways in a learning design. Esmonde (2016) discusses the relationship between power and artifacts, especially in a learning environment, and cites the example of curriculum standards and how their existence shapes the way K-12 teachers in the U.S. not only teach students, but how their performances are evaluated as well. Similarly, a learning design in an online environment in higher education generally includes predetermined learning objectives and disciplinary practices dictating how learners participate and develop their skills, or in other words, what types of practices and knowledge are considered valuable and worthy. Examining these arrangements of power and authority in a learning design is crucial to critical practice of instructional design.

Questions of identity and values are also essential to a productive critical instructional design effort. When considering identity and how it is relevant in learning, Vakil (2020) implores researchers to consider not only who students are in the present, but also who they have the potential to become as they learn. In practice, this means thinking about the diverse racial and gender identities taken up by learners and how those identities are reflected, or not, in an instructional design. Drawing on Wenger (1998), who stated, “learning is an experience of identity,” Vakil argues that as learners’ identities change, those shifts, “give rise to new ways of making meaning of and interacting with the world,” as well as their relationship with the world (p. 92). But throughout this process of shifting identity, learners are also considering what their disciplinary learning means for their “possible futures” (Vakil, 2020, p. 93). As a result, learners may question the narratives surrounding political and ethical values of their chosen disciplines, such whose perspectives are considered expert lenses in the field, leading learners to “consider the kind of person one has to be, or become, in order to participate in the communities of practice explicitly or implicitly associated with particular forms or domains of knowledge” (Vakil, 2020, p. 93). This can lead to both positive and negative impacts on learners’ disciplinary learning and their identities when those disciplinary values align or conflict, which makes ethical identity development and ethical values important commitments to this critical analysis.

To summarize, the extant literature points to three commitments, or for the purposes of this analysis, critical focuses, that create a map through this analysis. Taylor’s (2018) questions of representation, Esmonde’s (2016) ideas around power, and Vakil’s (2020) concept of disciplinary values serve as themes for exploring how a course design could be improved for more just outcomes.

A Framework for Critical Instructional Design

Using these critical focuses to guide the analysis, I also turned to Vossoughi and Gutiérrez (2016) to identify specific units of analysis within the design. In their discussion of critical pedagogy, Vossoughi and Gutiérrez (2016) challenge researchers to consider the “how” of teaching rather than “what is to be taught” (p. 143). This switch in focus calls for critical analysis of how learning is organized, how practices situate power and ideology, how social relations are established, and “how tools expand or limit opportunities for development of critical thought” (p. 143). Based on my experience as an instructional designer of online learning experiences, these units of analysis can be neatly aligned to the main component parts of a learning design. The way learning is organized could include not only the linear progression of a course as laid out in the syllabus, but also the learning management system (LMS) and choices about how learning is presented in a digital space with regard to features enabled or disabled in a given design. When considering power and ideology embedded in practices, I look to examine the activities and projects learners are completing. Social relations between instructor and learners, and between learners, can also be analyzed based on the practices included in a learning design and can be observed by looking at discussion posts and announcements among other artifacts. Examining the course policies and procedures around topics like late submissions and test taking can also answer questions regarding power and values. Finally, the tools, such as software, LMS integrations, and others, that are utilized both by the instructor and learners play an important role in online learning experiences and are important to examine in a critical analysis. These units of analysis identified by Vossoughi and Gutiérrez serve as focal points for uncovering data related to the critical focuses identified previously. The table below organizes this framework and includes the types of course data I will analyze. “Units of analysis” has become “sites of analysis” to clarify that those design components are where one should look for evidence related to the critical focuses. The following analysis examines a foundational course that serves as an entry point into a completely asynchronous bachelor’s degree program at a large research institution in the U.S. I used the master course space for the purposes of this work and investigated these areas of focus on my own in an effort to develop this framework, but in the future, this process could and should occur in partnership with other instructional designers and even faculty partners. Next, I will highlight specific examples from the focal course and expand on this framework, followed by suggestions for improvement or practices to consider replicating in other learning designs.

Table 1: Draft Critical Instructional Design Framework

Analysis

Organization

While many design components of any course would fit the organization category of analysis, for the sake of brevity this chapter will be limited to 1-2 components from each category. Of all the components that organize learning in the course, I focus here on the content. Since content itself is a broad term, this paper considers reference materials such as books, instructor-provided documentation, and examples of work as primarily composing course content.

The focal course is based in the arts and is made up of five lessons. Each lesson is structured to include reading a selection of chapters from the required text, and a book that is reviewed as a critical reflection on the role of the discipline in everyday life. The lessons also contain other reference materials in formats such as audio podcasts and videos. In addition, each lesson contains some form of practice activity and culminates in the completion of three projects. An outline of the topics covered in the course is included below.

• Design Thinking – systems thinking, critical thinking, design process
• Visual and Interaction Design – semiotics, inclusive design, critical design, visual design, identity design
• Storytelling – structure, development, character
• Open Design
• Self Design

At first glance, the organization of learning as it is reflected in these materials answers some of the guiding questions about disciplinary values and learner identity. Learners are introduced to the idea of using discipline-based insights as a frame to understand complex problems and develop comprehensive, critical solutions. As an introductory course that ushers students into the discipline at this university’s art school, the course topics prioritize key concepts alongside critical concepts relevant to the field, positioning these issues as foundational to the discipline. If they were not already, learners in this discipline will become practitioners who are at least made aware of the importance of these topics and will be expected to incorporate them into their practices within the course. A noteworthy observation about this course involves the perspectives from which these critical concepts are presented, a matter that will be addressed later.

Although I was not the instructional designer in charge of this course’s initial development, I am aware of the processes and decision making that went into it as a member of the unit within which the development was supported and this is where the questions of power regarding organization of learning are answered. In my department, the instructor is the ultimate authority on what content is included in a course, and in what order, which has the tradeoff of leaving out the voices of learners and other instructors within the department. This arrangement creates an additional barrier for instructional designers and our ability to make positive changes toward more equitable and inclusive instruction. When control over the course rests so singularly in the hands of one individual, there is a risk of replicating that instructor’s ways of knowing and being within a discipline, for better or worse. In the case of our focal course, the instructor chose to prioritize inclusive, critical discipline-relevant concepts alongside foundational disciplinary concepts, but not every individual instructor may have made that same choice. An area of improvement when it comes to power and authority in learning designs across the board would be to consider who the stakeholders are more broadly and include additional stakeholders in the process to ensure the voices of others are present in the course content. Additional stakeholders could be other faculty, a department chair, or even students.

Finally, representation in the organization of learning in the course can be analyzed by examining the authors who are cited in the course to learn who is present and who is missing. As part of an independent project, a past student kindly shared their data about representation in the course content which was collected, along with similar data from all the courses this student had taken so far in their program of study, for a project in another class. I verified these numbers and share them here with permission and gratitude to the student who chose not to be identified. Of the 59 scholars whose work appears in the course, 58 are white and of American or European descent, one is a Hispanic male, and seven are white women. To connect this oversight to an earlier point about power in the learning design process, bringing more voices into the process may have illuminated this lack of representation earlier in the process.

Social relations

Next, I will analyze how power and identity development are intertwined in the social relations supported in the learning design. Vossoughi and Gutiérrez don’t offer a specific definition of social relations in their chapter, so I lean on the instructional design definition of social interaction in online courses which centers on interaction among classmates and instructors. By this definition, there are two primary social relations that are specific to this class, which are learner-to-learner interactions and learner-instructor interactions.

There are many opportunities for learner-instructor interaction in the course design, as instructors provide feedback and grades for every activity learners complete, and learners are invited to use the commenting feature in the LMS to communicate with instructors regarding submissions. However, there is only one formalized opportunity for peer-peer interaction where learners can offer each other feedback and interact. Peer feedback in the form of studio critique has its own dedicated section in the course policies and procedures which outlines how feedback should be given and received, thus putting up boundaries around how students interact with each other during these activities. Such guidance cites the “critique sandwich” method of sandwiching suggestions for improvement between one to two positive comments. The critique guidance also offers tips on how to make this process useful: “If someone gives you ambiguous feedback, this means that they can intuitively see a weakness but might not know why something isn’t working. You should follow up with their comments with probing questions to better understand their perspective.” Learners are graded on critique as participation, so it can be assumed to be a disciplinary value in the design discipline, and learners could anticipate becoming comfortable with the idea of critiquing their peers with advice and criticism in the spirit of improvement.

Critique is a commonplace teaching strategy, especially in arts-based disciplines, but the process of giving and receiving feedback in this way is a practice entrenched in a culture that is worth questioning. The “critique sandwich” structure in particular does not allow for learners to express their positionality when providing a response to classmates’ work, but rather encourages students to make a value judgment on others’ work as good or bad, a practice with ties to white supremacy culture (Okun, 1999). While boundaries and structure have a beneficial place in a foundational course, an approach to critique that encourages appreciation for classmates’ efforts and recognizes there are a multitude of ways to provide substantive feedback could be a more productive method.

Critique sessions may be viewed as an opportunity for learners to demonstrate their identity development as an artist, and possibly distribute expertise among the class rather than looking at the instructor as the sole expert. Nowhere in the course is it explicitly stated that an instructor’s feedback during critique carries more weight or should be treated differently from peers, but since the instructor is the one handing out grades, it isn’t unreasonable to assume that instructor feedback may be perceived as non-negotiable. While the critique process may seem like a way for learners to practice being experts and hold some of the power within the course, ultimately the instructor likely maintains the most power in this activity as a result of setting the critique guidelines and also determining what an acceptable critique is. These social relations as designed are more related to identity development and practicing using disciplinary language and ways of interacting than they are designed to disrupt the power balance in the course.

Tools

The next focal point in this analysis is the tools utilized in the course learning design. Tools can both constrain and empower, and in an online learning experience, tools mediate learner’s interactions with learning materials, classmates, and their instructor. Since tools are so ubiquitous in online learning and mediate nearly all aspects of student learning, analyzing the values, power, and opportunities for representation that tools do and do not provide is essential to this work. There are several software tools learners utilize to complete projects, including InVision App, Twine, and Adobe Spark, which are available for use without purchase. The use of publicly available tools in a foundations course could convey to learners that specialty software is not necessarily required to do real design work, and dispel any potential notions that using higher cost tools produces higher quality work. On the other hand, these tools are very specific in nature and may constrain the projects learners produce in ways that are counterproductive to learning about design.

In addition to the software students use to produce projects, there are tools that students utilize to access course materials, submit assignments, and interact. This course is unique in that the learning design consists of three systems that work together to provide different functions. The LMS (Canvas) serves as the course home base and grading center because it is supported by the university and tied directly to the student information system which allows for seamless access without requiring students to create multiple accounts and provides customizable options for learners to set up notifications that remind of due dates and alerts for instructor communications. Tied into this system is the virtual studio tool, which sits outside of Canvas specifically for critique of in-progress work. The virtual studio tool was developed to provide a visual-based discussion area for students to share many different formats of work, such as audio/video files, large images, animations, etc. Third, the course uses GitHub to manage updates to the content and deliver course pages in order to maintain consistency across concurrent offerings. The GitHub site is accessed when learners click on the link from the course Canvas site and is positioned as an electronic textbook.

A critical analysis of the Canvas LMS could be its own book, so for the purposes of this analysis I will focus on the virtual studio. The studio primarily supports the disciplinary values of design by prioritizing visuals as the objects of discussion. Learners typically photograph their work from various angles (for physical objects) or upload a selection of in-progress screen shots for digital work, and provide a short artist statement for their classmates to accompany the images. This foundational course emphasizes the design process, so learners are encouraged to document their work at various stages and share it. Learners who may previously have only presented finished work in past educational experiences would ideally become people who are comfortable with the idea of work in progress through this sharing activity, and understand that projects are iterative and open to criticism and improvement.

In terms of “what matters” in the studio, comments and visuals are the primary features of the tool. The only options learners can utilize when they click on a peer’s work is to scroll through their images and add comments. By prioritizing the visuals and the comments submitted by learners, the studio reflects the values of looking at works in progress and considering feedback. As was mentioned earlier, the development of the virtual studio is another project that was primarily driven by one instructor, though more staff members were part of the process due to the complexity of skillsets required to complete this type of undertaking. Programmers, UX designers, and the instructional designer were all part of the team, but still student voices were absent from the development of this tool, even though it was designed specifically to support online learners’ studio activity. When using a specially-designed tool like this, learners’ activity could be constrained by the need to fit the submission model.

Practice

Finally, practice is the last unexplored site of analysis. When examining a learning design from an instructional design perspective, practices are considered to be the activities learners undertake that are guided by a set of goals. When it comes to the learning design of this class, components that I categorized as practices include the learning objectives, projects, and activities that learners participate in. While projects and activities may seem obvious, the inclusion of learning objectives as practices may initially be unclear, but I justify this choice because the learning objectives in any course ultimately guide the assignments, exams, projects and other activities learners engage in. Experienced instructional designers understand the learning objectives of a course to be an explicit statement of expectations for both the instructor and the learners. In an ideal world, the objectives are the guide posts that drive the learning experience, thus they are part of practices.

In addition to guiding learners’ progression, the learning objectives should also illuminate the disciplinary values of any course. For example, the focal course states the following objective: “Implement new ideas and develop a diverse array of options for problem solving in response to critical review and the iterative design process for improving work.” Learners might interpret from this objective what values are prioritized in the discipline and the course, which align with the values of iterative work, sharing, and openness to critique that I have uncovered in previous sections of this paper across the other sites of analysis.

I would also like to analyze a more traditional example of practice in the form of a project from the course, the Daily Design Journal. This is an introductory project learners begin at the start of the course, and the primary task is to document 14 objects learners encounter in their daily life. Learners are instructed to note the shape, form, materials, dimensions, functionality, and how those factors relate to the way they interact with the objects they choose. This activity is once again driven by the instructor, who sets the criteria for what matters and should be included in the final product, and also sets the tone for the reflective practice that follows the sketching portion where students respond to a series of questions about the three sketches of their choice. This activity doesn’t illuminate much with regard to representation within the course, but it does demonstrate another opportunity for learners to evaluate disciplinary values and practice those values by learning to think like a designer by examining designed objects and reflecting on how their designs connect with their functionality.

Discussion

I recognize this analysis as presented here is only the beginning of what will hopefully become a much more in-depth, refined process. I also hope this documentation serves as a starting point for re-evaluating our instructional design processes from the beginning of a course design, rather than retroactively considering issues of power, identity, inclusion, etc. When reflecting on whether the framework uncovered previously hidden problematic areas, it seems more like this version of the framework offered a way to articulate those issues. For instance, a theme that became painfully clear in this course, and likely would for many in my portfolio, is that there is only one subject matter expert typically present in the development, and that leads to issues of power and representation that have been outlined above. While many may have assumed this would be a problem, I did not imagine the breadth and depth of the influences power can have on all aspects of the course. I went into this work assuming the critique activity was an opportunity for learners to have some power in the course, but in reality, my reflections within the other sites of analysis made it abundantly clear that this activity is not at all balanced due the instructor-imposed guidelines on interactions.

Another valuable outcome of this effort is disrupting the notion that I could break each piece of the course apart and analyze them individually. I quickly realized that this framework, at least as I interpreted it, leads to a lot of overlap and influences other parts of the framework. For instance, the course procedures and policies dictate a lot of the social relations in the course. Participation grades in particular mean there has to be a designed interaction, and similarly, that need affects the practices that are designed into a course, which in turn influences the way learning is organized. All of these factors are constrained by the tools available. The choices made about course designs early in the process and the beliefs held by instructors and designers have a waterfall effect that touches nearly every experience designed into a course. This leads to the power problem I referred to that continually resurfaces throughout each of my chosen sites of analysis. It is the decision-making of the person or people in power that influences who is represented and what disciplinary values are prioritized, and how learners may develop new identities within these experiences. As a result, this discovery calls for further reflection around the notion of power and whether it should be positioned differently in the framework, or addressed in an entirely different way. In addition, the influence of power in our work leads to the question of what instructional designers can do to shine a light on this issue.

Reflecting on the critical focus of disciplinary values and identity development, I wonder now if these are two different categories. To elaborate further, alignment of disciplinary values is good from an instructional design perspective, but that doesn’t necessarily mean the values themselves are inherently good. Those responsible for the learning design need to determine together if the values represent ethical outcomes for all, or if the values conflict. Additionally, learners also need to decide for themselves if the values they interpret are in alignment with their own personal values, regardless of the ethics in the disciplinary values. Further research into how students interpret disciplinary values, as well as how disciplinary experts interpret disciplinary values, could help this part of the analysis. Perhaps instructional designers can design opportunities for students to examine their personal values against the perceived values of their chosen area of study throughout their college careers.

After this initial trial, I also wonder if there is a better way to consider questions of representation in this framework? Looking at who is present in the referenced materials was productive, but I struggled to identify any other instances of representation in the practices, tools, or social relations within the course. There are undoubtedly different ways to think about representation that I have overlooked in this work as a result of my privileged way of experiencing education, and exploring research around heterogeneity in learning might benefit the next iteration of this framework. This gap may also require further consideration of stakeholders and how they think about ideas of representation and identity within the discipline, and whether the course design meets those expectations.

This work is limited in some ways. While I hope the framework will help drive critical analysis of existing learning designs across all disciplines, I am biased by my experience as an instructional designer who has primarily worked in professionalized disciplines. The three themes or critical focuses I outlined at the beginning of this paper may not be as functional for learning designs in fields with less direct applications of skills and knowledge. Another bias that may have unintentionally impacted this framework is my experience in online learning design, which is much deeper than my experience in traditional residential learning design.

In addition, this framework was influenced only by the critical scholarship that I have encountered. Undoubtedly there are other authors and papers unknown to me whose work may strengthen or contradict this framework. I hope this framework sparks conversation and collaboration around iterations for future use to evaluate other courses in my portfolio, and hopefully with instructional designers from other disciplines interested in this line of work. As I emphasized above, when one individual holds all the power, systemic problem areas remain undiscovered. The experiences of other professionals can only strengthen this effort.

Conclusions: The Instructional Designer’s Role in Critical Instructional Design

Near the beginning of this paper, I cited Barab et al., who discuss the role instructional designers have in educational designs and the responsibilities that should be carried with this role. The authors state that “designers should regard their work in terms of its impact not on a situation directly but, rather on how users transact with the work, with each other, and with their contexts” (Barab, et al., 2007, p. 296). As an instructional designer, exploring critical scholarship in the learning sciences has changed my perspective on what my role is when collaborating with instructors and other team members to develop learning designs. I have always viewed this role as one of advocacy; for pedagogy, for innovative and memorable learning experiences, and for learners in the margins. While that hasn’t changed, this analytical exercise has shown that previous ideas of who is in the margins are not sufficient. Based on my analysis of this course alone, I already feel that I should advocate for change in the process itself by ensuring there are voices other than mine and the instructor I’m working with involved when we are making these design choices. As I discussed previously, power relations are abundant and tangled within the typical instructional design process, and drawing attention to this problem should be the goal of all instructional designers.

To borrow once again from Collier (2020), a “small move” I can make now could be working with my team to remedy the problematic spots highlighted in this analysis, and encouraging my teammates to apply this process to their own portfolios. Sharing our discoveries could help our unit find ways to embed these critical focuses from the beginning of a new project in the future, and help us make a plan for revising our old designs. Additionally, finding ways to bring more people into the design process can be another “small move” to tackle in the present. Specifically, striving to include student voices in the instructional design process in ways that are authentic and meaningful for both parties will be an immediate focus in my professional work going forward. I can advocate for spreading out the power in the instructional design process, ask the right questions, and point to the literature. By adopting this critical stance, I can be one of those instructional designers who takes steps to “build transformative models of what could be” (Barab, et al., 2007, p. 264).

References

Barab, S., Dodge, T., Thomas, M. K., Jackson, C., & Tuzun, H. (2007). Our designs and the social agendas they carry. Journal of the Learning Sciences, 16(2), 263–305.

Collier, A. (2020). Inclusive Design and Design Justice: Strategies to Shape Our Classes and Communities. EDUCAUSE Review. https://er.educause.edu/articles/2020/10/inclusive-design-and-design-justice-strategies-to-shape-our-classes-and-communities.

Costanza-Chock, S. (2020). Design Justice: Community-led practices to build the worlds we need. Cambridge: MIT Press.

Esmonde, I. (2017). Power and sociocultural theories of learning. In I. Esmonde & A. N. Booker (Eds.), Power and privilege in the learning sciences: Critical and sociocultural theories of learning (pp. 6–27). New York, NY: Routledge.

Okun, T. (1999). White supremacy culture. White Supremacy Culture. https://www.whitesupremacyculture.info/

Taylor, K. H. (2018). The role of public education in place-remaking: From a retrospective walk through my hometown to a call to action. Cognition and Instruction, 36(3), 188–198.

Vakil, S. (2020). “I’ve always been scared that someday I’m going to sell out”: Exploring the relationship between political identity and learning in computer science education. Cognition and Instruction, 38(2), 87-115.

Vossoughi, S. & Gutiérrez, K. (2017). Critical pedagogy and sociocultural theory. In I. Esmonde & A. N. Booker (Eds.), Power and privilege in the learning sciences: Critical and sociocultural theories of learning (pp. 139–161). New York, NY: Routledge.

Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge, MA: Cambridge University Press.

Winner, L. (1986). Do artifacts have politics? In The whale and the reactor (pp. 19-39). Chicago: University of Chicago Press. https://ebookcentral.proquest.com/lib/pensu/detail.action?docID=557593