EVALUATE: Understanding and Skills

Distinguishing Between Information Resources

Information is published in a variety of formats, each with its own special considerations when it comes to evaluation. Consider the following formats.

Social Media

Social media is prominent in the landscape of information gathering. Facebook updates, Instagram posts, tweets, wikis, Tik Toks, YouTube, and blogs have made information creators of us all. These platforms have a strong influence on the way we gather and disseminate information. Social media is often the first place we learn about current events or discover new items of interest. Anyone can create or contribute to social media, and there are no gatekeepers to fact check for accuracy or censor slanderous or hateful speech.

Information Scientists might use social media for the aggregate data that is collected from users to gain insights, but this requires special training. Most other academics use social media as primary source accounts of the human experience around historic events or social phenomena.  You might look for the social media posts of the politically powerful, intellectually important or socially influential to gain their insights. Eyewitness accounts and reactions may also be important sources of information for context.

News Articles

These days, social media platforms are the first place many people  get information about a big news story. News media often post to social media as they learn new facts and insights. Longer form news articles and reports are generated quickly as the story progresses. News articles are written by journalists who report on an event they have witnessed firsthand, or after they make contact with those more directly involved. Journalists focus on information that is of immediate interest to the public and they write so that a general audience can understand the content.

In research, news articles are often best treated as primary sources, especially if they were published immediately after a current event.

Magazine Articles

While news articles and social media tend to concentrate on what happened, how it happened, who it happened to, and where it happened, magazine articles tend to explore why something happened, usually with the benefit of a little hindsight. Writers of magazine articles also fall into the journalist category and rely heavily on investigation and interviews for research. Fact-checking in magazine articles tends to be more accurate because magazines publish less frequently than news media, so they have more time to get the facts right. Depending on the focus of the magazine, articles may cover current events or just items of general interest to the intended audience. The language may be more emotional or dramatic than the factual tone of news articles, but the articles are written at a similar reading level to appeal to the widest audience possible. Magazines can be a more accessible resource to explore why something happened.

STEP 1: Choosing Materials

When choosing a source for your research, what criteria do you usually use? Gauging whether the source relates to your topic at all is probably one. How high up it appears on the results list when you search may be another. Beyond that, you may base your decision at least partly on how easy it is to access.

While these may be important criteria, there are other criteria you should keep in mind when deciding if a source will be useful to your research.

Quality

Clues to a source’s level of quality are closely related to thinking about how the source was produced, including what format it was published in and whether it is likely to have gone through a formal editing process prior to publication.

Scholarly journals and books are traditionally considered to be higher quality information sources because they have gone through a more thorough editing process that ensures the quality of their content. Generally, you also pay more to access these sources or may have to rely on a library or university to pay for access for you.

Accuracy

A source is accurate if the information it contains is correct. Sometimes it’s easy to tell when a piece of information is simply wrong, especially if you have some prior knowledge of the subject. But if you’re less familiar with the subject, inaccuracies can be harder to detect, especially when they come in subtler forms such as exaggerations or inconsistencies.

To determine whether a source is accurate, you need to look more deeply at the content of the source, including where the information in the source comes from and what evidence the author uses to support their views and conclusions. It also helps to compare your source against another source.

Relevance

Does the source relate to your topic and, if it does, how closely does it relate? Some sources may be an exact match; for others, you may need to consider a particular angle or context before you can tell whether the source applies to your topic. When searching for relevant sources, you should keep an open mind—but not too open. Don’t pick something that’s not really related just because it’s on the first page or two of results or because it sounds good.

You can assess the relevance of a source by comparing it against your research topic or research question. Keep in mind that the source may not need to match on all points, but it should match on enough points to be usable for your research beyond simply satisfying a requirement for an assignment.

Bias

An example of bias is when someone expresses a view that is one-sided without much consideration for information that might negate what they believe. Bias is most prevalent in sources that cover controversial issues where the author may attempt to persuade their readers to one side of the issue without giving fair consideration to the other side of things. If the research topic you are using has ever been the cause of heated debate, you will need to be especially watchful for any bias in the sources you find.

Bias can be difficult to detect, particularly when we are looking at persuasive sources that we want to agree with. If you want to believe something is true, chances are you’ll side with your own internal bias without consideration for whether a source exhibits bias. When deciding whether there is bias in a source, look for dramatic language and images, poorly supported evidence against an opposing viewpoint, or a strong leaning in one direction.

Reputation

What is the reputation of the author and the publication in which the source appears? Is the author a professor? A blogger? If the author is a professor, are they respected in their field or is their work heavily challenged?

Digging a little deeper to find out what you can about the reputation of both the author and the publication can go a long way toward deciding whether a source is valuable.

You can investigate the reputation of an author by looking at any biographical information that is available as part of the source. Looking to see what else the author has published and whether this information has positive reviews is also important in establishing whether the author has a good reputation. The reputation of a publication can also be investigated through reviews, word-of-mouth by professionals in the field, or online databases that keep track of statistics related to a journal’s credibility.

Credibility

Is the source believable or trustworthy based on the evidence such as information about the author, the reputation of the publication, how well “put together” the source is – these factors all contribute to whether a source has credibility. How likely would you be to use a source that was written by someone with no expertise on a topic or a source that appeared in a publication that was known for featuring low quality information? What if the source was riddled with spelling and formatting errors? Looking at sources like these should inspire more caution.

Objectively, credibility can be determined by taking into account all of the other criteria discussed for evaluating a source. Knowing that some types of sources, such as scholarly journals, are generally considered more credible than others, such as self-published websites, may also help. Subjectively, deciding whether a source is credible may come down to a gut feeling. If something about a source doesn’t sit well with you, you may decide to pass it over.

Evaluating your source’s sources

Evaluating the sources you use for quality, accuracy, relevance, bias, reputation, and credibility is a good first step, but have you ever thought about evaluating the sources used by your own sources? This takes extra time, but looking at the reference list, bibliography, or notes section of any source you use to gauge the quality of the research done by the author of that source can be an important extra step.

STEP 3:  Analyzing Information Resources

Key Takeaway

Keep in mind that the overall goal of analysis is not just to present a collection of gathered evidence.  The goal is to strategically gather evidence that will answer your research question.

Once you have identified relevant articles, books, websites and sections of books, you need to do a more in-depth analysis of your resources. Analysis can help you find answers to your research questions, solutions to your problems, and logical arguments to organize your thoughts. This examination of specific resources is the most time-consuming part of your research process because it requires careful reading of your resources. Often this examination is done while you are developing your final research project.  To do this effectively, you need to take time to read and examine resources.

Knowing When to Stop

For some researchers, the process of searching for and evaluating sources is a highly enjoyable, rewarding part of doing research. For others, it’s a necessary evil on the way to constructing their own ideas and sharing their own conclusions. Whichever end of the spectrum you most closely identify with, here are a few ideas about the ever-important skill of knowing when to stop.

• You’ve satisfied the requirements for the assignment and/or your curiosity on the topic.
  • If you’re doing research as part of a course assignment, chances are you’ve been given a required number of sources. Novice researchers may find this number useful to understand how much research is considered appropriate for a particular topic. However, a common mistake is to focus more on the number of sources than on the quality of those sources. Meeting that magic number is great, but not if the sources used are low quality or otherwise inappropriate for the level of research being done.
• You have a deadline looming.
  • Nothing better inspires forward motion in a research project than having to meet a deadline, whether it’s set by a professor, an advisor, a publisher, or yourself. Time management skills are especially useful, but since research is a cyclical process that sometimes circles back on itself when you discover new knowledge or change direction, planning things out in minute detail may not work. Leaving yourself enough time to follow the twists and turns of the research and writing process goes a long way toward getting your work in when it’s expected.
• You need to change your topic.
  • You’ve been searching for information on your topic for a while now. Every search seems to come up empty or full of irrelevant information. You’ve brought your case to a research expert, like a librarian, who has given advice on how to adjust your search or how to find potential sources you may have previously dismissed. Still nothing. It could be that your topic is too specific or that it covers something that’s too new, like a current event that hasn’t made it far enough in the information cycle yet. Whatever the reason, if you’ve exhausted every available avenue and there truly is no information on your topic of interest, this may be a sign that you need to stop what you’re doing and change your topic. If you are working on an assignment for class, you should discuss a change of topic with your professor.
• You’re getting overwhelmed.
  • The opposite of not finding enough information on your topic is finding too much. You want to collect it all, read through it all, and evaluate it all to make sure you have exactly what you need. But now you’re running out of room on your flash drive, your Dropbox account is getting full, and you don’t know how you’re going to sort through it all and look for more. The solution: stop looking. Go through what you have. If you find what you need in what you already have, great! If not, you can always keep looking. You don’t need to find everything in the first pass. There is plenty of opportunity to do more if more is needed!

Applying Theory to Practice

Evaluation is arriving at a conclusion of the quality of something (whatever research you’re using).

Analysis is the in-depth study of the material to better understand the information being presented. Evaluation is subjective; analysis is objective.

Now that you know more about the theory, it’s time to apply the theory.

The following section will help you put source evaluation into perspective using something called the CRAAP Test. The CRAAP Test is a list of questions to help you evaluate the information you find. Not all criteria apply equally to all articles but will give you confidence that your sources meet the expectations of your assignment. (Evaluating)

 

The CRAAP test is a series of common evaluative elements you can use. The test was developed by librarians at California State University at Chico and it gives you a good, overall set of elements to look for when evaluating a resource. Let’s consider what each of these evaluative elements means.

 

CURRENCY: Assessing currency means understanding the importance of timely information

Certain topics require you to pay special attention to how current your resource is—because they are time sensitive, because they have evolved so much over the years, or because new research comes out on the topic so frequently.

• • • • • When was the information published or posted?
        • Has the information been revised or updated?
        • Does your topic require current information, or will older sources work as well?
        • WEB SOURCE: Are the links functional?

 

RELEVANCE: The importance of the information for your specific needs.

Understanding what resources are most applicable to your subject and why they are applicable can help you focus and refine your thesis. Many topics are broad and searching for information on them produces a wide range of resources. Narrowing your topic and focusing on resources specific to your needs can help reduce the piles of information and help you focus in on what is truly important to read and reference.

• • • • • Does the information relate to your topic or answer your question?
        • Who is the intended audience?
        • Is the information at an appropriate level (i.e. not too elementary or advanced for your needs)?
        • Have you looked at a variety of sources before determining if this is one you will use?
        • Would you be comfortable citing this source in your research paper?