16.1 Rhetorical Source Evaluation

The Rhetorical Analysis chapter shows how to conduct a rhetorical analysis of a particular rhetorical artifact, identifying its structure in terms of purpose, audience, writer, and studying the way it uses kairos, ethos, logos, and pathos to achieve its purpose. Why shouldn’t we use those same tools to evaluate sources we want to use for our own purpose in the argumentative research essay?

Well, there is no reason we shouldn’t, and in fact you may have already done it using the CRAAP method, which you will see in the chart below is just a collection of questions that are ultimately asking the same questions we asked of our rhetorical artifact in our rhetorical analysis:

• Who is the author?
• What it the purpose?
• Who is the audience?
• How does it use ethos, pathos, logos, and kairos either singly or in combination to achieve its purpose?

Any source you consult for your argumentative research essay is, in essence, a rhetorical artifact. When you approach these artifacts for the purposes of evaluation and not just analysis, suddenly your purpose and how it relates to the purpose of the artifact in question becomes critical.

16.1.1 Analysis vs. Evaluation

According to the Oxford English dictionary, to analyze something is to “identify and describe [its] component parts” (“analyze”), which is what we did when we wrote our rhetorical analyses. When you evaluate something, you are certainly doing analysis, but you are also deciding as to its value, in addition to identifying and describing. The rhetorical analysis makes a determination as to the effectiveness of the rhetorical artifact it focuses on, but it stopped short of making any value judgement.

Now that you are going to use rhetorical concepts to evaluate, you have to consider when and why a source is valuable to you, and this depends very much on how the purpose of the source aligns with your purpose as a writer.

16.1.2 Source Purpose: The Rhetorical Modes

In the rhetorical analysis, you analyzed sources for one purpose: argument. One of the main ideas was to assume that the purpose of the rhetorical artifact you studied was to persuade someone of something. In fact, many scholars argue that anything that communicates anything is making an argument. We make arguments every day without even realizing it, like when we decide on the clothes we are going to wear, for instance (See Lunsford 1). Even so, scholars, writers, and teachers of rhetoric have consistently broken writing down into the following categories, or modes:

• Narration (to tell a story)
• Description (to describe something)
• Exposition (to explain)
• Argument (to persuade)

You may encounter sources that have some of these other purposes, and more importantly, you may find that they are valuable to you in terms of supporting your argument. For instance, if you are arguing for more open immigration policies, you may find a story (narration) about a child that was separated from her parents and left alone in a foreign country due to unnecessarily harsh immigration policies. Or, conversely, if you are arguing for more stringent immigration policies, you might find a story about a dangerous criminal who could stay in a country despite being there illegally. Neither of these stories may be an argument, but they can be used as support for your argument, and that is what makes them valuable for your purposes. Note that using either of these stories in support of the argument, by themselves, would constitute a hasty generalization fallacy, so that determination can become part of the evaluative process.

The point is, evaluation is the process of deciding 1) is the source intrinsically valuable (is it a good story? a truthful story? etc.), and 2) Does it align with your purpose.

16.1.3 Your Purpose: BEAM

Sometimes a source may not be intrinsically valuable or credible, but still may be valuable for your research. For example, let’s say you were writing a paper about rhetoric, and you wanted to show an example of a logical fallacy like the hasty generalization. It would then be necessary to include a source that is intrinsically not credible. This kind of source is an example, and it is perfectly acceptable to have it among your list of sources, and talk about it in your inquiry research journal, if you discuss why you are using it.

Joseph Bizup of Columbia University invented another useful acronym for different uses you might have for sources, or different evidence: BEAM. BEAM stands for background, exhibit, argument, and method, all of which are legitimate uses you might have for sources. See the explanation below:

• Background: using a source to provide general information to explain the topic. For example, the use of a Wikipedia page on the Pledge of Allegiance to explain the relevant court cases and changes the Pledge has undergone.
• Exhibit: using a source as evidence or examples to analyze. For a literature paper, this would be a poem you are analyzing. For a history paper, a historical document you are analyzing. For a sociology paper, it might be the data from a study.
• Argument: using a source to engage its argument. For example, you might use an editorial from the New York Times on the value of higher education to refute in your own paper.
• Method: using a source’s way of analyzing an issue to apply to your own issue. For example, you might use a study’s methods, definitions, or conclusions on gentrification in Chicago to apply to your own neighborhood in New York City. (“Beam”)

16.1.4 CRAAP, TRAAP, and A CRAB – What’s the Difference?

You have probably stumbled across the CRAAP method of evaluating sources. Along the way, someone might have introduced you to the TRAAP method and the A CRAB method. Each of these methods is just a recombination of the same concepts. Currency in the CRAAP method is the same as Timeliness in the TRAAP method. Otherwise, relevance maps to relevance, authority to authority, and so on.  You can see how the words formed by the acronyms here form their own argument. CRAAP is a reference to Ernest Hemingway’s claim that “the most essential gift for a good writer is a built-in, shock-proof, shit detector. This is the writer’s radar and all great writers have had it.”^[1] TRAAP is just a variation on the same idea, that to get tricked by sources that lack credibility or use tricky rhetoric is to get caught in a trap. Now A CRAB – I’m not sure there is an argument implicit in that one. There might be, I just don’t see it.

For more advanced academic writing, it might be time to move past the acronyms to the bigger picture (although if any one of these acronyms is helpful to you, keep using it!). What do they all have in common? The rhetorical framework we used to analyze rhetorical artifacts provides just such an overview.

If an acronym helps, call it the KELP PAW method, Where:

K = Kairos

E = Ethos

L = Logos

P  = Pathos

P = Purpose

A = Audience

W = Writer

There is an implicit argument here, too, if you agree that kelp is a weed and it needs to be kept out of the best argumentative research essays or it will turn into a kelp monster.

Let’s leave it at that. If you like the acronym, use it. If not, just remember the core concepts or use another acronym you like better to remember them. Take a look at the chart below to get an idea of how the CRAAP method compares to the rhetorical method.

16.2 The Rhetorical Source Evaluation Method

Note that many of the questions below are taken directly from the Lumen: English Composition II – Rhetorical Methods Based section on the CRAAP method and re-categorized under different headings. Consulting a list of questions like this can be extremely helpful, but as you begin to see the organizing principle behind them, you might be able to come up with questions of your own.

16.2.1 Rhetorical Context

While the “rhetorical context” refers to purpose, audience, and writer, it also includes a general context you should be mindful of that might include answers to the following general questions:

What is the historical background of the conversation this source is engaging in?

What is the cultural context of the conversation this source is engaging in?

What is the economic context of the conversation this source is engaging in?

What is the political context of the conversation this source is engaging in?

16.2.1.1 Writer

Who is the author?

Why did I choose this source?

Do my search terms reveal any hidden agenda that might indicate I am choosing only sources that agree with my point of view (confirmation bias)?

16.2.1.2 Purpose

Why does this information exist?

What does it want its audience to think/do?

Is it to persuade? To tell a story? To describe? To explain?

Does the home page or a clear link from the home page reveal the author’s or sponsor’s motivation for establishing the site?^[2]

How does the information relate to my topic or research question?

What argumentative purpose could this source serve for me? (think BEAM: background, exhibit, argument, or method)?

16.2.1.3 Audience

Who is the intended audience?

Do the appeals this source uses align with the beliefs, values, opinions, motivations, and concerns of my audience?

16.2.2 Rhetorical Appeals

16.2.2.1 Kairos

When was the information published or posted? Has it been in the past 10 years?

Is it recent enough to be relevant to your thesis? Why or why not?

Is it the most recently updated version of the information or is it the original? Depending on your purpose, either one of these could be more desirable.

How recently has the website been updated? Has it been within the last 3 years? (web)^[3]

16.2.2.2 Ethos

16.2.2.2.1 Situated Ethos

• What are the credentials of the author?
• What organizations is the author associated with?
• Is the document author or site sponsor clearly identified? (Web)^[4]
• Does the site identify the occupation, position, education, experience, or other credentials of the author? (Web)^[5]
• Does the site provide contact information for the author or sponsor such as an email or organization address? (Web)^[6]
• Is the source biased in any way, and if so, does it acknowledge that bias?
• Is the source’s angle of vision broad or narrow?
• Does the source engage in unnecessary and frequent attacks on the character of other authors?

16.2.2.2.2 Invented Ethos

• Is the source completely free of distracting grammatical errors or proofreading oversights?
• Does the source use formatting consistent with its genre?
• If it is an academic source, does it use a recognized style and documentation format appropriate to the discipline it exists in (MLA, APA, Chicago)?
• Is the source visually appealing, or does it appear to have been thrown together quickly without regard for visual design?

16.2.2.2.3 Borrowed Ethos

What types of sources are referenced?

• Do the referenced sources represent the existing conversation fairly, or do they all seem to come from a particular point of view or angle of vision?
• Are the referenced sources themselves credible?
• Are the sources appropriately cited in the text and listed in the references?

16.2.2.3 Logos

• Can the same information be found in other reliable sources?
• If the source is an experiment, can it be replicated?
• Does the source contain any logical fallacies?
• Is the documentation provided in support of the information substantive?
• Can the facts cited by the source be verified elsewhere?
• Are the facts provided by the source cited in context?
• When citing facts that are contested, does the source acknowledge the controversy, or does it cite them as if they were universally agreed upon?
• Does the source use statistics reasonably?
• If my source is statistical information, does it avoid hasty generalizations by ensuring sufficiently large sample sizes?
• Are quotations in context, so that the original meaning is retained? Can you trace them to their original context?
• What is omitted or censored from this text?^[7]

16.2.2.4 Pathos

• Does the source overuse appeals to pathos?
• If the source is a story, does the pathos support my argument?
• If the source is a story, and the pathos supports my argument, will including it present any problems with hasty generalization?
• Are there other, similar stories that can be used to show this is not an isolated incident?
• Is the degree of advocacy of this source unnecessarily extreme?
• Does the source contain fallacies of pathos (appeal to popularity or the bandwagon effect, appeal to stirring symbols, appeal to pity)?