Introduction to Chapter 16

Imagine that you have been asked to write an article for your university newspaper about what we’ve learned in the 30 years since the Chernobyl Disaster (when an explosion in 1986 at nuclear power plant in the Soviet Union released radioactive particles into the air over much of the USSR and western Europe). You’ve already done some basic research on Wikipedia, then used Google scholar to investigate some more of the health effects. Finally, you searched in your university health, science, and medicine databases to learn specifics about the impact of the disaster.

How do you know which sources are worth using in your article? How will you know if the sources are even good? Journalists famously cover the 5 W’s (who, what, where, when, why…and how) in their articles, and these similar questions can be used to evaluate your search results:

• Who: Who is the author and what are his/her credentials in this topic?
• What: Is the material primary or secondary in nature?
• Where: Is the publisher or organization behind the source considered reputable? Does the website appear legitimate?
• When: Is the source current or does it cover the right time period for your topic?
• Why: Is the opinion or bias of the author apparent and can it be taken into account?
• How: Is the source written at the right level for your needs? Is the research well-documented?

If you can answer all of these questions, you’ll understand more about the quality and usefulness of a source for your article.

In this section, you’ll learn more about tools like this that help you examine the usefulness and appropriateness of information for your research. You’ll use the CRAAP test to evaluate a source and consider techniques to help you synthesize pieces from multiple sources in your writing.

16.1     Evaluating Sources

Critical thinking is interwoven in all steps of the research process, and one of the earliest places you will use it is when you collect and evaluate your sources. You have already begun collecting sources for your project, and perhaps you even have a sense of which sources are going to be the most useful. The credibility of your research paper is a function of your sources. If you consult scholarly sources in your field, you will have a better understanding of your issue and provide a well-supported, respectable position. If, on the other hand, you consult only “soft” source material (magazines, for example), your research is going to lack the depth it needs to be convincing.

The two main questions you should ask yourself when evaluating sources are the following:

1. Is this source suitable?
2. Is this source trustworthy?

Not every suitable source is trustworthy, and not every trustworthy source is suitable.

16.2     Determining Suitability

Your task as a researcher is to determine the appropriateness of the information your source contains, for your particular research project. It is a simple question, really: will this source help me answer the research questions that I am posing in my project? Will it help me learn as much as I can about my topic?  Will it help me write an interesting, convincing essay for my readers?

Here are some reasons to include information:

• contains facts/opinions that you need
• contains illustrations or data you need
• contains an overview to establish the context of your paper
• was written by a well known authority or expert
• contains a point of view that illustrates something you are trying to establish
• exemplifies something
• may have a clear explanation of something

Reasons to exclude information:

• it may not be from a scholarly journal
• it may be from a scholarly journal but be too difficult for you to understand
• it may be out of date
• it may not have the point of view you are researching
• it may not contain any new information.
• it may be too narrow (or too broad) in coverage

16.3     Determining Trustworthiness

To determine the trustworthiness of a source, you want to ensure that a source is current, written by an expert, accurate, and unbiased. You’ll want to consider the rhetorical context of a source, including its purpose, audience, and focus.

One excellent tool to examine both the reliability and trustworthiness of a source is the CRAAP method, which stands for:

• Currency: the timeliness of the information
• Relevance: the importance of the information for your needs
• Authority: the source of the information
• Accuracy: the reliability, truthfulness, and correctness of the information
• Purpose: the reason the information exists

Let’s take a closer look at how analyzing the CRAAP in a source can serve as a valuable source evaluation tool.

Currency: The Timeliness of the Information

Key Question: When was the item of information published or produced?

Determining when an item of information was published or produced is an aspect of evaluating information. The date information was published or produced tells you how current it is or how contemporaneous it is with the topic you are researching. There are two facets to the issue of currency.

• Is the information the most recent version?
• Is the information the original research, description, or account?

The question of most recent version of information versus an original or primary version can be a critical one. For example: If you were doing a project on the survival of passengers in car crashes, you would need the most recent information on automobile crash tests, structural strength of materials, car wreck mortality statistics, etc. If, on the other hand, you were doing a project on the feelings of college students about the VietNam War during the 1960s, you would need information written in the 1960s by college students (primary sources) as well as materials written since then about college students in the 1960s (secondary sources). Key indicators of the currency of the information are:

• date of copyright
• date of publication
• date of revision or edition
• dates of sources cited
• date of patent or trademark

Relevance: The Importance of the Information for Your Needs

Key Question: How does this source contribute to my research paper?

The discussion of suitability above is essentially the same thing as relevance. When you read through your source, consider how the source will effectively support your argument and how you can utilize the source in your paper. You should also consider whether the source provides sufficient coverage of the topic. Information sources with broad, shallow coverage mean that you need to find other sources of information to obtain adequate details about your topic. Information sources with a very narrow focus or a distinct bias mean that you need to find additional sources to obtain the information on other aspects of your topic. Some questions to consider are:

• Does the information relate to my topic or answer my question?
• Who is the intended audience?
• Is the information at an appropriate level (i.e. not too simple or advanced) for my needs?
• Did I look at a variety of sources before deciding to use this one?
• Would I be comfortable using this source for my college research paper?

Authority: The Source of the Information

Key Question: Is the person, organization, or institution responsible for the intellectual content of the information knowledgeable in that subject?

Determining the knowledge and expertise of the author of information is an important aspect of evaluating the reliability of information. Anyone can make an assertion or a statement about some thing, event, or idea, but only someone who knows or understands what that thing, event, or idea is can make a reasonably reliable statement or assertion about it. Some external indications of knowledge of or expertise are:

• a formal academic degree in a subject area
• professional or work-related experience–businessmen, government agency personnel, sports figures, etc. have expertise on their area of work
• active involvement in a subject or organization by serious amateurs who spend substantial amounts of personal time researching and studying that subject area.
• organizations, agencies, institutions, corporations with active involvement or work in a particular subject area.

HINT: Be careful of opinions stated by professionals outside of their area of work expertise.

Accuracy: The Reliability, Truthfulness, and Correctness of the Information

Key Question: How free from error is this piece of information?

Establishing the accuracy, or relative accuracy, of information is an important part of evaluating the reliability of information. It is easier to establish the accuracy of facts than it is opinions, interpretations, or ideas. The more an idea, opinion, or other piece of information varies from the accepted point of view on a particular topic the harder it is to establish its accuracy. It may be completely accurate but corroborating it is both more necessary and more difficult. An important aspect of accuracy is the intellectual integrity of the item.

• Are the sources appropriately cited in the text and listed in the references?
• Are quotations cited correctly and in context? Out of context quotations can be misleading and sometimes completely erroneous.
• Are there exaggerations, omissions, or errors? These are difficulty to identify if you use only one source of information. Always use several different sources of information on your topic. Analyzing what different sources say about a topic is one way to understand that topic.

In addition to errors of fact and integrity, you need to watch for errors of logic. Errors of logic occur primarily in the presentation of conclusions, opinions, interpretations, editorials, ideas, etc. Some indications that information is accurate are:

• the same information can be found in other reliable sources
• the experiment can be replicated and returns the same results
• the documentation provided in support of the information is substantive
• the sources used for documentation are known to be generally reliable
• the author of the information is known to have expertise on that subject
• the presentation is free from logical fallacies or errors
• quotations are “in context”-the meaning of the original work is kept in the work which quotes the original
• quotations are correctly cited
• acronyms are clearly defined at the beginning

Some indications that information may not be accurate are:

• facts cannot be verified or are contradicted in other sources
• sources used are known to be unreliable or highly biased
• bibliography of sources used is inadequate or non-existent
• quotations are taken out of context and given a different meaning
• acronyms are not defined and the intended audience is a general one
• presence of one or more logical fallacies
• authority cited is another part of the same organization

Purpose: The Reason the Information Exists

Key Question: Who is this information written for or this product developed for?

Identifying the intended audience of the information or product is another aspect of evaluating information. The intended audience of an item generally determines the style of presentation, the level of technical detail, and the depth of coverage. You should also consider the author’s objectivity. Are they trying to persuade? Do they present any bias? While it is unlikely that anything humans do is ever absolutely objective, it is important to establish that the information you intend to use is reasonably objective, or if it is not, to establish exactly what the point of view or bias is. There are times when information expressing a particular point of view or bias is useful, but you must use it consciously. You must know what the point of view is and why that point of view is important to your project. For example, books on food sanitation written for children, for restaurant workers, or for research microbiologists will be very different even though they all cover the same topic.

Determining the intended audience of a particular piece of information will help you decide whether or not the information will be too basic, too technical, too general, or just right for your needs. The intended audience can also indicate the potential reliability of the item because some audiences require more documentation than others.

For example, items produced for scholarly or professional audiences are generally produced by experts and go through a peer evaluation process. Items produced for the mass market frequently are not produced by experts and generally do not go through an evaluation process. Some indications of the intended audience are:

• highly technical language, complex analysis, very sophisticated/technical tools can indicate a technical, professional, or scholarly audience
• how-to information or current practices in “X” are frequently written by experts for practitioners in that field
• substantive and serious presentations of a topic with not too much technical language are generally written for the educated lay audience
• popular language, fairly simple presentations of a topic, little or no analysis, inexpensive tools can indicate a general or popular audience
• bibliographies, especially long bibliographies, are generally compiled by and for those doing research on that topic

THE CRAAP TEST

Review the CRAAP method and practice evaluating sources in this tutorial from Eastern Michigan University.

Be sure to complete the practice exercises at the end of the tutorial.

Licenses and Attributions

 CC Licensed Content, Shared Previously:

• Introduction and sections 16.1, 16.2, an 16.3 were adapted from Source Analysis by Lumen Learning. License: CC BY-NC-SA: Attribution-NonCommercial-ShareAlike