A paper I wish I had read 5 years ago: How to Read a Paper

As a graduate student, one of the most fundamental skills that I must develop is the ability to read and digest research papers.  However, this critically important skill is rarely taught in any explicit way, students are expected to pick it up on their own as they progress through graduate school (1).  So if I could step into a time machine and go back to when I began my PhD, I would get myself to read the following paper: S. Keshav, How to Read a Paper (SIGCOMM 2007). It details a three pass approach for reading and understanding research papers, that goes like this:

  • First Pass: (5 – 10m)  
    1. Read the title, abstract, introduction.
    2. Read the section and sub-section headings.
    3. Read the conclusions.Examine the references, noting any that you have already read.
  • Second Pass:  (1h)
    1. Read the remaining sections, but skip any technical sections such as proofs.
    2. Examine the figures carefully.  Scrutinize small but important details like axes, error bars, and any statistical methods used to justify the validity of the conclusions.
    3. Mark any relevant but unread references for further reading.  Ideally each of these should be given a first pass reading.
  • Third Pass: (5h+)
    1. Spend as much time as required to thoroughly read and understand the paper, to the point that you could virtually re-implement the contents yourself.
    2. Imagine yourself as starting from the same assumptions made by the authors, and then think about how *you* would argue to support the same conclusions.  The comparison between the original arguments and your parallel arguments should be enlightening.  Maybe you will spot an implicit assumption made by the original authors.  Maybe you will find that the argument is more general than is stated by the original authors, and that some assumptions they made can be relaxed to give a more widely applicable method.  In any case, if you reach the beginning of the third pass and see fit to put in the effort required to re-implement the ideas in the paper, you will almost surely learn something new in doing so.

One of the nice aspects of this method is that it fails early, so that you don’t have to spend more than 5 to 10 minutes on a paper that will not be relevant to your own research.  Another aspect is that it can improve your own writing: if papers are reviewed this way, then you should really pay attention to how your introduction and conclusion read, as well as your section headings.  If a knowledgeable reader in your field cannot skim and summarize your paper based on these alone, your paper is probably going to be rejected.  Finally, for students who are embarking upon a literature survey, this methods provides a valuable road-map for how to use Google Scholar or CiteSeer to quickly read and digest a lot of papers in your research area.  While I wish I had read this years ago, at least I’ve found it now.  I’m sure it will save me lots of valuable time in the future.  Thanks to Yashar Ganjali for mentioning it in his graduate research skills seminar.

[1] This is the case with other critical skills, such as the ability to read and understand code, as others have pointed out.  It may even exist for a good reason that I have yet to understand.  It is sometimes also the case that these skills *are* presented informally, as part of graduate student seminars (this is the case here at University of Toronto).