Monday, November 10, 2014

A Formula for Academic Papers: Related Work


The Related Work section of an academic paper is often the section that graduate students like writing the least. But it is also one of the most important sections to nail as the paper heads out for review. The Related Work section serves many purposes, several of which relate directly to reviewing:
  • The person handling the submission will use the referenced papers to identify good reviewers,
  • Reviewers will look at the references to confirm that the submission cites the appropriate work,
  • Everyone will use the section to understand the paper's contributions given the state of existing research, and
  • Future researchers will look to the Related Work section to identify other papers they should read.

Framing

When placing your research in the context of the existing literature, there is no need to show that prior work is all wrong to show that your paper makes a contribution. In fact, doing so is likely place reviewers and readers who have written relevant content in a combative frame of mind as they read. But these are the people you most want to listen to what you have to say! So rather than focusing on why your work is better than what has already been done, show how it builds on existing knowledge to provide additional insight. Write your Related Work section as if you were telling the cited authors why they should care about the work that you present. After all, they are the people who will probably be reviewing your paper.

Papers do not need to exist in a vacuum to be interesting, and the typical contribution that can be made in ten pages is relatively limited. While you should aim to change the world with your body of research, you are unlikely to do so within a single paper. Avoid over stating your contributions. Be particularly wary of declaring that your paper is the "first paper that we are aware of" in an area. While I know it is tempting to do so (and I am even guilty of doing it myself), statements like this usually trigger an intense related work search when I see them as I review. It should concern you if you believe nobody else has looked at an analogous problem to yours. Chances are very high that there is something similar out there that your work can draw from.

Conversely, do not freak out if you happen to come across related work during your literature search that seems to address the exact same problem that you are studying. In an ideal world you would identify all relevant work prior to starting your own research so that it can inform your approach, but in the real world that doesn't always happen. You may have holes in your initial lit review, or related work may be published after you begin a study. Surprisingly, however, this is generally a good thing. It means that other people are excited and interested in your topic, and it provides you with a fresh way of looking at things. There is a large space to study surrounding any problem, and your work probably makes a contribution. The challenge is just to figure out what that is and how to communicate it clearly to your readers. 

Because reviewers will be drawn in part from the papers you cite, cite papers written by people you would like reviewing your work. Reviewers are likely to look at your citation list to ensure completeness -- and, sometimes, to ensure that papers they have written are cited. To avoid bruised egos, do not leave significant holes and try to include papers by a variety of different authors.

Include citations to your own papers when relevant, even if the paper you are writing is being submitted anonymously. There is no need to anonymize these citations. Instead, cite your papers the same way you would cite any other paper, in the third person. For example, in a paper on cross-session search I write, "Teevan et al. [32] showed, via query log analysis, that nearly 40% of queries were attempts to re-find previously encountered results." But while it is fine to cite your own work, be wary of over-citing yourself. Too many papers by an unexpected person typically signals that that person is an author, and generally looks bad.


Structure

A typical Related Work section follows a basic structure:
  • It starts with few sentence overview of the general space, and
  • A preview of areas that are particularly relevant and will be discussed in detail.
  • The body consists of several paragraphs, each discussing a different relevant thread of research.
  • The section ends with a paragraph summary of the paper's contributions over existing research.
The first paragraph of the Related Work section should help a knowledgeable reader in the field place the work into a general bucket. Be aware of what is common knowledge in the community where the paper will appear to avoid giving too sweeping of an overview. There is, for example, no need to explain that there has been a lot of research into web search at the start of an information retrieval Related Work section. Instead, highlight the specific sub-area and tell the reader what aspects of that sub-area are particularly relevant.

In the body of the Related Work section, do not just list paragraphs that each summarize a single related paper. Summaries can be a useful way for you to build a picture for yourself of existing related work. But in the Related Work section, you should help your reader get the lay of the land by grouping and organizing the existing research. Start each paragraph with a sentence describing why the papers discussed in that paragraph are related, citing all of the papers to which the criteria applies. Then write a sentence or two about several of the most relevant papers from the group, highlighting the approach used and relevant findings. End the paragraph with a sentence explaining how the work in your paper contributes something new in light of these papers.

An example of this basic structure for a paragraph in the body of a Related Work section can be found in A Crowd-Powered Socially Embedded Search Engine:
ACM papers use numbers to cite related work, which provides limited context compared to other citation formats. Do not treat these numbers as part of the paper's text. You should not, for example, say, "In [23] the authors explore ..." Instead, write the text as if the numbers weren't there, saying, "Teevan et al. [23] explore ..."

End the Related Work section with a paragraph that summarizes what is know given existing literature, and highlight why the work to be presented in your paper offers a valuable contribution beyond this. An example can be found in Understanding How the Projection of Availability State Impacts the Reception of Incoming Communication:
In summary, the work presented in this paper builds on previous research to explore how availability information relates to people’s communication decisions. While earlier work focused on how availability information impacts the people initiating communication, we focus on its impact on the decisions of the recipient. Further, we are able to study this behavior at a much larger scale than previously possible by looking at the users of a popular enterprise communication system that infers its users’ availability.

Length

Even though the Related Work section is very important to the overall paper, it should not be too long. It a ten page paper, a good rule of thumb is that you should be done with the paper's set-up (including the Introduction and Related Work sections) and on to the meat of the paper by the start of the third page. A good target length for a Related Work section in standard ACM format is one to two columns. To keep the section short, avoid subsections unless really necessary.

The actual list of references at the end of the paper should also be compact. I generally aim to fit the Conclusion and reference list entirely within a single final page. One way to do this is to target a reasonable number of references. I've averaged 31 references across all 10 page papers I've written since 2010. While you should not shrink the font size of your reference list, there are tricks you can play to minimize the amount of space each individual reference takes. For example, you can refer to common, well known outlets by their acronym. Do not, however, be sloppy with how you refer to papers. This can offend the paper authors and make it hard for readers to find the associated papers. Include correct, consistent details for each. Be sure to list references in alphabetical order to make it easy for the reader to scan to see if a paper is cited, since this is a task your reviewers are likely to do several times while reviewing.

Some communities place the Related Work section at the beginning of a paper, while others place it at the end. When writing a paper you should follow the norms of the community where you are publishing. However, if there is a choice about placement I recommend putting the section at the beginning. When I read papers that do not cover related work until after the main content, I find I spend much of my time reading wondering how what is being presented fits into the bigger picture.

Hopefully this post has convinced you to be thoughtful with how you place the research papers you write in context. The Related Work section is an extremely important part of the paper, and the resulting citation graph helps define the structure of the field. As a bonus, here are links to a few papers we have written that take advantage of the citation graph to make it easier to explore related work:

---

This post is part of a series of posts about the formula for academic papers. The components being discussed are:

15 comments:

  1. Great post and great advice, Jaime!

    Adding to what you said, I'm curious what you think about when the Related Work section should be written? In particular, I'm wondering if you think it's a useful idea to write a draft of it very early so you are clear about what people have already done, what hasn't been done, and exactly how other attempts to do something like what you are doing or answer the question you are answering differed? In drafting the related work early, you can discover work you didn't know about and end up changing your approach based on what others already found, shoulders of giants and all that?

    The reason I ask is Related Work is usually one of the first sections I draft, mostly for the reasons above, but you said graduate students hate writing this section, which might imply they do it last. Id love to hear your thoughts on that if you don't mind expanding on it a bit, thanks!

    ReplyDelete
    Replies
    1. Good question, Greg -- although timing questions are hard! Keeping on top of relevant literature before, during, and after a research project is important. There are many different ways to do this, including: running literature searches, carrying over knowledge from previous searches, attending conference, sharing your research and getting pointed to relevant content, reviewing, and writing the Related Work section. Personally, I almost never actually WRITE the literature review until fairly late in the process. The points I am trying to make in any particular paper just evolve too much in the process for me to be able to do it at the beginning.

      Delete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Editing is becoming popular among youngster because they have needed it a lot. The edit personal statement will enable them to recorrect their statement and remove all mistakes.

    ReplyDelete
  4. That's so nice information, thanks to share this awesome post with us, this technique is really so superb or useful for us...I feel great to find all these information at in this blog...!!
    best dissertation assistance

    ReplyDelete
  5. Zotero is a free tool to collect, share, organize and cite your research sources.

    ReplyDelete
  6. This comment has been removed by a blog administrator.

    ReplyDelete
  7. This comment has been removed by the author.

    ReplyDelete
  8. For students, especially PhDs it's very important to write research papers according to latest trends and standards.
    So first you have to think on what are you going to write and start writing after creating strict plan

    ReplyDelete
  9. This comment has been removed by the author.

    ReplyDelete
  10. This comment has been removed by the author.

    ReplyDelete
  11. This is the core of your research papers. Your methodology should be crisp, clear and definitive. Research paper

    ReplyDelete
  12. This was a satisfying one.For getting a pleasurable job you compulsion to have a enjoyable resume.So, you dependence to create a sample professional resume. To know about science article review

    ReplyDelete
  13. Nice post.Thank you,I found the best resource to update my knowledge.Hope this information can help me to crack GIS jobs in Hyderabad.

    ReplyDelete