Download PDF of  How to Read a Scholarly Paper

How to Read a Scholarly Paper

Cavendish McKay  |    Marietta College  |  December 4, 2017

General Notes

Academic (and in particular, scientific) writing is not like other types of writing you may have encountered in the past. This is due in part to the fact that when scholars write for an academic audience, they can assume a different level of preparation on the part of their readers than someone who is writing a book or article for more general consumption. Part of your goal when approaching a piece of academic writing is to ask yourself, “Am I prepared to read this?”

Academic writing tends to be information-dense, and therefore effective academic reading is hard work. Two tools make the job a little easier: taking notes while you read, and making multiple, focused passes through the material. This document is intended to help you focus your reading so that you can get the most out of a piece of writing with as little frustration and pain as possible.

PASS 0: “Getting Acquainted”

Primary questions

• What is this piece of writing I have in front of me?
• Is it reputable?
• (Perhaps most importantly) How long is it going to take me to read this?

Notes

Your zeroth pass through the material is purely to become acquainted with it. To do so, you should make note of

• The title
• The author(s), including affiliation(s)
• The source (Journal, preprint server, etc.)
• Length
• Are there figures/pictures? If so, what kind, and how many?
• What kind and how many (roughly) references are there? If an article doesn’t have references, it isn’t a piece of academic writing.

You should complete this pass by making an outline of the structure of the document. In most cases, this will look like a list of sections. For many journal articles, this will actually tell you surprisingly little about the content of the article, but we’re not yet to the point of trying to understand content, so that’s okay.

After this pass

Let’s get quantitative. Make a guess (at this point, it’s still very much a guess) at how long it is going to take you to read this paper. Write your guess down. Then time each subsequent pass through the material to evaluate the quality of your guess. As you practice this, your guesses will get better. This is an extremely valuable skill for managing your study time, especially if and when you get to the point of doing independent research.

PASS 1: Abstract

Just as references are basically a universal feature of academic writing, so too is the abstract. Longer form writing (such as books) won’t usually have an abstract per se, but the same idea could be applied to a preface (and this document isn’t really directed at book length documents, anyway). The abstract is supposed to tell you what the paper is about, and why you should care.

Primary questions

• Why should I care about this?
• Is it likely to be relevant to what I’m trying to learn/do?

Notes

Read the abstract. When you’re done reading, rewrite the abstract in your own words, using the simplest possible language. Focus on these two questions:

• What specific question or questions does this paper answer?
• What specific claim or claims does this paper make?

Additionally, this is the time to start building a list of jargon used in the paper. For our purposes, jargon is any word you don’t understand, either in meaning or usage. Most fields use at least some seemingly simple words in very specific (and sometimes nonintuitive) ways. The ability to correctly understand and use jargon is one trait that marks you as an insider within a particular field. Jargon is useful because it serves as a kind of shorthand — it packs a great deal of meaning into not a lot of space. If you are going to extract meaning from this article, you will need to be able to parse the jargon it uses. Make a list of all of the terms in the abstract that you don’t understand. Include words whose meanings you think you know, but which are being used in a way that seems odd to you. Don’t bother hunting for definitions at this point; the abstract doesn’t usually define terms. You will use this list in your next passes through the material.

After this pass

Record how long it took you to read the abstract. Note that all of the above note-taking should be included in your reading time! Compare this time to both the length of the abstract and your reading time estimate from pass 0. Revise your estimate (if necessary). Assess: Is this article going to be useful to me? If it is something that was specifically assigned for you to read as part of a class, the answer is “yes.” If it was given to you by your advisor as introductory material, the answer is also “yes.” See if you can figure out why you ought to be interested. If this is an article that came to you as a part of your own research, this question takes a little more thought.

PASS 2: Introduction

Almost every paper has an introduction. It’s purpose is to let you know what you need to know to be able to sucessfully read the paper. It obviously can’t cover everything (recall Carl Sagan’s statement that to bake an apple pie from scratch, you must first create the universe), but in general, jargon is defined and a basic set of premises for the current work is laid out.

Primary questions

• Do I already know enough to read this paper and make sense of it?
• What is the world view that underlies this paper?

Notes

As you read the introduction, you should be maintaining two distinct lists. The first is a continuation of the jargon list you started in the abstract. It contains all terms that are unfamiliar to you. Now, however, you should be able to either give a definition to each term in your list or at least connect it with a reference to another paper. The second list contains specific claims made in the introduction, and the specific references that are tied to them.

After this pass

Again, record the amount of time you spent reading the introduction, compare with your estimates, and adjust as necessary.

Assess: Am I prepared to get what I need from this article? If there is a lot in the introduction that you didn’t understand, then you probably need some more background work for this paper to be as useful to you as it could be. If you have the time and opportunity, you can use the list of claims and references you made in this pass to generate (or add to) a reading list. If your reading list gets long enough that you know you won’t have time to get through it, you can use the topics to find another resource (such as a textbook, for example) that may be more accessible. If this paper is assigned reading for a course or research project, and you feel that you won’t be able to make sense of it, your notes from the introduction can serve as the basis for a conversation with the person who made the assignment. You might start the conversation by saying something like: “In reading the introduction for this paper, I felt like I will need to understand (x, y, z from your list). What is the best way for me to get from my current level of understanding to where I need to be so that I can get what I need from this paper?”

PASS 3: Conclusion/Discussion

After you have read the introduction and have the context, you should skip to the end and read the conclusions (if there are any) and discussion. The purpose of these sections is to concisely but completely lay out the claims that were referenced in the abstract. This is the heart of the paper—what is original and important in the work being reported.

Primary questions

• What’s the point? (i.e., what are the author’s trying to prove?)
• What did they do to prove it?
• Did they succeed?

Notes

Continue your list of jargon. If the paper is reasonably well written and you have been thorough in your earlier passes, there should be relatively little new jargon in this section. It is likely, however, that you will need to refer to your jargon list to understand the claims being made.

As you did in both the abstract and the introduction, make a list of specific claims being made in this section, as well. These claims should be supported not by external references (as they were in the introduction), but by data from earlier in the paper. Make notes of what you expect to see in the data, and what the authors say the data shows.

If there are a number of claims in this section, try to figure out which one(s) the authors think are most important.

After this pass

Write, in your own words, what the authors think they proved in this paper.

Assess: Does this seem believable? You might categorize the conclusions on a five-step scale:

1. Totally obvious
2. Feels right, but nice to have proof
3. I hadn’t thought about it, but don’t see any reason to believe otherwise
4. Counterintuitive
5. Seems impossible

PASS 4: Results/Data

How closely you examine the data will depend at least in part by how much you need to be convinced to accept the claims. In an ideal world, we might be just as skeptical of claims made in support of things we already believe as we are of claims that contradict our beliefs, but humans don’t seem to work like that. As a consequence, if you found the conclusions compelling before looking at the data, you should force yourself to exercise extra attention in

this pass.

Primary questions

• Are the conclusions supported by data?
• What are the sources of uncertainty or error?

Notes

In most cases, data will be reported in tables or graphs. It doesn’t make sense for you to reproduce these in your notes (especially true of tables; drawing a sketch of a graph might be useful in certain circumstances), but you should try to interpret them in your notes.

For each figure/table:

• Make a note of its label (e.g. “Figure 3” or “Table 1.1”)
• Summarize the caption
• In your own words, tell the story encountered in the data
• Check the author’s version of the story for this data. Does it match your version?
• Are uncertainties reported?
• How big are the uncertainties? How do they affect the story?
• If the data sets are statistical in nature, how big are the sample sizes? What standard is used for significance?
• To which of the claims (from the previous pass) does this data apply?

After this pass

Assess: did the data adequately support the claims made? Pay special attention to claims unsupported by data, or data that isn’t tied to a specific claim.

PASS 5: Methodology

Generally speaking, the methodology section is most useful if you are trying to run an experiment of your own that can be directly compared with the results of the paper. One could argue that the methodology section should expose flaws in experimental design or execution, and while I agree in principle, my personal belief is that in practice there’s too much uncertainty

about how closely the procedure as written down matches the procedure as carried out for this to work well. A better way of checking for methodological problems is to actually try to replicate the study.

That said, reading methodologies in published work can give a newcomer to a particular field a window on what is accepted or standard practice within that field. This is true of both  experimental procedures and data analysis methods.

Primary questions

• Could I reproduce this research given the description?
• What methods/standards were used (in general terms)

Notes

If you are not already an expert in the field, it is going to be difficult for you to find any errors in methodology, so don’t focus on that. Instead, pretend you want to replicate the study. Write out in brief terms what you would have to do to reproduce the research presented in the paper. Make special note of anything that seems to be missing. Don’t worry if you can’t immediately find anything; missing information is hard to find until you actually start trying to reproduce someone else’s experiment.

After this pass

You’re essentially done, unless you want to try to reproduce the author’s results.

Assess: Do I want to try to reproduce the author’s results? How would I go about doing so?

Conclusion

Reading academic writing is difficult and time consuming, but necessary and ultimately rewarding. Your effectiveness will be greatly enhanced by

• taking multiple, focused passes through the material
• taking notes while you read

Hopefully the guidelines in this document will be useful to you as you continue in your academic efforts.

Download PDF of  the Worksheet for  How to Read a Paper

Worksheet: How to Read a Scholarly Paper

PASS 0: Identify the following for this paper

Title:

Author(s):

Author Affiliations:

Source (journal, etc.):

Length (pages):                                     Are there pictures or figures?  YES      NO

If so, what kind and how many?

How many references are there?

How long do you estimate it will take to read this paper? 

PASS 1: Abstract

Read the abstract and answer the following:

How long did it take you to read the abstract?

Is this relevant to what I’m trying to learn or do?

What “jargon” is used in the abstract? (meaning any words you don’t understand)

Now, rewrite the abstract in your own words in the simplest possible language, making sure to include:

1. what questions does this paper answer, and
2. what claim or claims does the paper make?

Based on the abstract, is the article going to be useful to you?

PASS 2: Introduction

Read the introduction. How long did it take to read?

What additional “jargon” is used in the introduction? (meaning any words you don’t understand)

Are you prepared to get what you need from this article? Can you understand enough about it?

Pass 3: Conclusion/Discussion

Read the conclusion/discussion. How long does it take to read?

What was the author trying to prove in this paper?

What did they do to prove it?

Did they prove it?

What additional “jargon” is used in the conclusion? (meaning any words you don’t understand)

Write in your own words what the author(s) thinks they proved in this paper.

Does the author’s conclusion seem believable? Rate it as:

1. Totally obvious
2. Feels right, but nice to have proof
3. I hadn’t thought about it, but don’t see any reason to believe otherwise
4. Counterintuitive
5. Seems impossible

Overall, how long did it take to read all sections of the paper?

How close is that to your estimate?

This worksheet supplements “How to Read a Scholarly Paper.”

Download PDF of the CRAAP Test Evaluation Rubric

CRAAP Test Evaluation Rubric

Use this guide to help determine a source’s CRAAP score using the rubric point system. point system.

Rubric Score Instructions

1. Enter information about the source at the top of the page, i.e. title, url, author, dates
2. For each line, starting with Currency, read each box from left to right and choose the one that matches your source the best
3. In the right-hand column, enter the column number, 1-4, that corresponds to the description that best matches your source
4. After every line has a score, tally the numbers in the right-hand column and write the total at the bottom of the page
5. The total score is out of 24 total points. A “good” score for a source to be used in a college research paper is between 20 and 24. You must determine the lowest score you will accept.

CURRENCY

• When was the information published or posted?
• Has the information been revised or updated?
• Does your topic require current information, or will older sources work as well?
• WEB SOURCE: Are the links functional?

RELEVANCE

• Does the information relate to your topic or answer your question?
• Who is the intended audience?
• Is the information at an appropriate level (i.e. not too elementary or advanced for your needs)?
• Have you looked at a variety of sources before determining if this is one you will use?
• Would you be comfortable citing this source in your research paper?

AUTHORITY

• Who is the author / publisher / source / sponsor?
• What are the author’s credentials or organizational affiliations?
• What qualifies the author to write about this topic?
• What affiliations does the author or organizational affiliate have? Could these affiliations affect their position?
• What organization or body published the information? Is it authoritative? Does it have an explicit position or bias?
• WEB SOURCE: Does the URL reveal anything about the author or source? examples: .com .edu .gov .org .net

ACCURACY

• Where does the information come from?
• Is the information supported by evidence? Is the source well-documented? Does it include footnotes, citations or a bibliography?
• Can you verify any of the information from another source or from personal knowledge?
• Has the information been reviewed or refereed?
• Does the language or tone seem unbiased and free of emotion?
• Is the information written clearly and free of typographical and grammatical mistakes? Does the source look to be edited before publication?
• WEB SOURCE: Is the information crowd sourced or vulnerable to changes by other authors, i.e. Wikipedia or other public wiki?

PURPOSE

• What is the author’s purpose? Is it to inform, teach, sell, persuade, or entertain?
• Do the authors / sponsors make their intentions or purpose clear?
• Is the information fact, opinion, or propaganda?
• Is the article presented from multiple points of view?
• Are there political, ideological, cultural, religious, institutional or personal biases?
• Is the information clearly supported by evidence?

The C.R.A.A.P. Test was created by University of California at Chico, Meriam Library; this rubric is based on that content with some modification.

CRAAP Test Evaluation Rubric Worksheet

Author(s):                                                         Publish Date/Last Update:
Title:
Volume & Issue# (for periodicals):

Publisher Name/Organization:
Publisher Location (for print & ebooks):
URL/Permalink (for electronic resources):
Retrieval Date (for electronic resources):