Digital Composition and Multimodal Texts

What Are Multimodal Texts?

Multimodal texts utilize sensory elements to further their rhetorical purpose and persuade an audience. These elements can include audio, visual, and/or physical. You can create a multimodal text using a digital technology tool, but you can also create a multimodal text by hand as well. The following are some examples of digital multimodal composition:

• Infographic
• YouTube video
• Podcast
• Website
• Blog
• Text message
• Word Document

And since we’re looking at types of multimodal texts, let’s also look at some examples of multimodal composition that take place outside of the digital realm:

• Collage
• Poster
• Speech
• Sculpture
• Painting
• Architectural models

The following video, developed for the Arizona State University writing program,  discusses how multimodal composition applies to first-year  composition courses.

Why Use Digital Composition and Multimodal Texts?

Since our world communicates through mostly digital means, learning how to compose in a digital environment is key to your success not only in school, but also in your current and future jobs, and as a member of society.

In addition to multimodal assignments preparing you for your future writing endeavors, multimodal assignments also allow students to use what they know. Melanie Gagich in her essay “An Introductionto and Strategies for Multimodal Composing” writes that students already have knowledge of multimodal composition. She writes in her essay, “Understanding that you are already composing multimodally in many digital spaces will help you transfer that knowledge and experience into your academic assignments. This understanding might also help alleviate any fears or anxiety you may have when confronted with an assignment that disrupts what you think writing should look like” (Gagich 74). If creating a multimodal text seems new to you, chances are you already have an applied understanding of multimodality, just by virtue of living and socializing in the 21st century.

Gagich continues and writes:

Gagich writes that there are five steps to creating a multimodal text:

1. Determine your rhetorical situation.
2. Review and analyze other multimodal texts.
3. Gather content, media, and tools.
4. Cite and attribute information appropriately.
5. Begin drafting your text.

Making Visual Choices

According to Foundations of Communication, it may be a cliché to say, “A picture is worth a thousand words,” but visual images have power. Good communication is a multisensory experience. Pre-literate children gravitate toward books with engaging pictures. As adults we graduate to denser books without pictures, yet we still visualize ideas to help us understand the text. That’s because a strong image in a poem or a story appeals both to the readers’ senses and emotions or intellect. Advertisers favor visual media—television, magazines, and billboards—because they are an effective way to hook an audience. Websites rely on color, graphics, icons, and a clear system of visual organization to engage Internet surfers. Visuals bring ideas to life for many readers and audiences in multiple ways:

• As a link between raw data and usable knowledge
• To provide concrete, vivid, and quick representations
• To save space
• To speak in a universal language
• To be persuasive

Types of Visuals

There are many types of visuals you can incorporate in digital and physical multimodal composition to illustrate and emphasize your point. The rest of this section describes how visuals can support and enhance your ideas in a multimodal text.

Symbols

Symbols include a range of items that can be either pictographic or abstract. In the image above, mathematical symbols and the image of a heart are used to convey the concept of love.

It’s a visual way to represent love, which is an abstract noun that means different things to a wide range of people.

Maps

Maps sometimes include map charts, or statistical maps.In the image to the right, two human palms are displayed, and a map of the world is painted onto them. In the background there is a blue sky with white, fluffy clouds. A map can represent more than geography. This map is a representation of our world, but the image could also speak to lines drawn by humans. One ironic feature of this picture is that the world’s territorial lines are drawn over the lines of two human hands.The image could also speak to matter, how nearly three-fourths of the world is made of water, and how up to 60% of the human body is made of water. Could the image speak to the interconnectedness of all beings on earth? That’s up to the reader to interpret. For more information on how to develop an interpretation you can support in your writing, read over Chapter 22 Using Interpretation to Develop Thesis.  Images can also persuade. For more information on how to analyze visual rhetoric, check out Chapter 24 Analyzing Visual Rhetoric.

Graphs and Tables

Graphs can take a variety of forms, the most common being line graphs, bar graphs, and pie charts.

In the graphs above, the creator is using visual representations of numbers to represent growth and decline in their topic. Graphs are a helpful way to visually illustrate change.

Diagrams 

This visual illustrates a process. One example of a diagram would be a flow chart.  The diagram below illustrates a workflow process.

Photographs

Photographs (still or moving) depict concrete objects, tell a story, provide a scenario, and persuade an audience. 

In the image to the right, a picture of a baby is displayed clearly on a smart phone in the foreground. In the background of the picture, an elderly woman smiles, and her face is blurry. There is contrastbetween the baby whose image is clear, and the woman whose image is blurred. The contrast piques the viewer’s interest. Contrasting images, colors, and subjects can draw a reader into an image force them to ask questions. For more information on using contrast in images, read over Chapter 18.2.

Illustrations

Illustrations can berealistic or abstract.The illustration pictured to the right displays a cartoonish picture of a Polaroid camera, the iconic camera from the eighties. For some readers, the illustration may invoke nostalgia, while for younger viewers, the camera may have a slightly historical feel. The reception of the illustration varies depending on the audience; regardless, the illustration can help persuade a wide range of audience members.

Why Use Visuals?

There are a number of reasons you might consider including visuals in documents, presentations, and other communications. Four reasons are detailed below:

Decorative: Visuals that do not represent objects or actions within the text but are added, instead, for aesthetic effect are considered decorative. Decorative visuals are often added to gain attention or increase the audience’s interest. Visuals can be used this way but can detract from the message you are trying to communicate and, thus, should be used with caution.

Representational: These visuals physically represent or physically resemble objects or actions in the text and are relevant to the content of the text. For example, rather than giving a detailed textual description of a new playground, you might include an image or render of the new playground and use the text to highlight specific features or information.

Analogical: Analogical visuals are used to compare and contrast two things, and explain their likeness or correspondence. For example, a marketing consultant might try to clarify the difference between targeted marketing and mass marketing by including images of a single fisherman with a single fishing rod and line next to an image of a bigger boat with a fishing net. By using the fishing analogy, the marketing consultant is attempting to connect possible prior understanding of the audience, a visual, and the concepts of targeted marketing versus mass marketing.

Organizational: The purpose of organizational images is to provide structure to information, visually define relationships, and illustrate connections. A chart of the hierarchical structure of a company is one example of an organizational image.

Table 18.1 Communication Purposes and Visuals

There are many considerations to keep in mind when choosing visuals. When possible, use a variety of types of visuals, but remember that any visuals you use should enhance the content of the text. For example, only add photos if viewing the photos will clarify the text. Near each visual, explain its purpose concisely. Do not expect your readers to figure out the values of the visuals on their own.

For repositories of openly licensed photos, you can search Wikimedia Commons and Pixabay. Website creation software like WordPress, social media applications like Twitter, and other types of software like graphics makers such asCanvaand video creators like Powtoon have their own repositories of free images you can use as well.

This section, “Visual” adapted from “A Picture is Worth 1000 Words: Using Visuals” in Part 1 of Foundations of Professional Communication CC BY 4.0Originally published at http://www.procomoer.org/foundations/

Making Audio Choices

Including audio in your multimodal project can enhance your text and move a reader both logically and emotionally. Audio enhances your message. You may want to consider the following audio choices:

• Music
• Spoken word
• Sound effects

Music: From pop to classical to Bollywood, music can be any use of vocalizations or instrumentals. Music can help convey theme in a video or podcast, and music can help heighten tension and advance plot in a story.

Spoken word: Spoken word audio choices include recording a voice over of the written text or a narration.

Sound effects: Sound effects include any kind of sound, from nature or manmade. A couple of examples include crickets, glass shattering, or applause. Sound effects can help characterize people and convey action. Sound effects can also affect the tone of a text, creating humor or suspense.

National Public Radio develops a wide range of podcasts that integrate music, narration, and sound effects to to tell stories. The sample podcast episode aired on the NPR show Hidden Brain.

Why Use Audio?

There are a number of reasons you might consider including audio in documents, presentations, and other communications. According to the Advisory Group on Computer Graphics (AGOCG), using audio in multimedia has the following advantages:

The AGOCG also writes that there can be disadvantages to using audio:

• Like most media, files can be large.
• Audio can be easily overused, and when sounds are continually used users tend to tune them out.
• For most people, audio is not as memorable as visual media.
• Good quality audio can be difficult to produce, and like other media most commercial audio, particularly music, is copyright.
• Users must have appropriate hardware and software.

Accessibility

Being a digital writer means that you have to be consciously aware of your audience and their ability or inability to participate in the texts that you create. Not everyone can view a meme or infographic and not everyone can hear the sound on a YouTube video.

Closed Captions

Whenever possible, include captions for all videos that you create. This allows those who are hearing-impaired and deaf to access your message. You can edit the videos yourself to add captions, or you can use a platform like YouTube that will auto-generate captions that you can edit.

Audio Description

If you are making a video that contains scenes with any type of action, you will want to create an audio description.

Transcript

For any type of audio or video, you want to consider including a transcript of the spoken dialogue.

Alternative Text

For any photo or graphic you include in your text, make sure to provide alternative text by right clicking on the image in Word, and then select edit Alt-text. using Alt-text is a principle of web accessibility. Users with screen readers will be read an alt attribute to better understand an on-page image.

Key Takeaway

The purpose of this chapter is to acquaint students with multimodal and digital composition. The following sections of chapter 18 explore the process and tools you can use while digitally composing texts for your academic and workplace audiences.

This chapter is a synthesis of two different creative commons texts:

Sections are written by Anne Turner, published by Central New Mexico Community College, and licensed under CC BY-SA 4.0

Making Visual Choices was adapted from “A Picture is Worth 1000 Words: Using Visuals” in Part 1 of Foundations of Professional Communication CC BY 4.0 Originally published at http://www.procomoer.org/foundations/

Works Cited

Advisory Group on Computer Graphics, “Using Audio in Multimedia,”

Gagich, Melanie. “An Introduction to and and Strategies for Multimodal Composing Melanie Gagich.” Writing Spaces: Readings on Writing, Volume 3, licensed under CC BY-NC-ND 4.0, https://writingspaces.org/sites/default/files/1gagich-introduction-strategies-multimodal-composing.pdf