This is the second in a planned series of editorials covering all aspects of good science writing.
Science can be thought of as the combination of three essential things: 1) a communal collection of knowledge (both facts/data and theories); 2) a method of evaluating the efficacy of scientific theories by comparing the predictions of those theories to observation/experiment; and 3) an attitude of skeptical inquiry and the belief that all scientific knowledge is provisional and subject to revision when confronted with new evidence. (A popular alternate breakdown of the “norms” of science, emphasizing its sociological nature, is Merton’s “cudos,” first introduced in 1942: communalism, universality, disinterestedness, originality, and skepticism.1) This breakdown of science into a body of knowledge, a method, and an attitude is useful in assessing the “scientific” content of any given behavior. If any one of these three pillars of science is missing from an activity, one cannot claim that that activity is scientific.
The growth of scientific knowledge is predominately incremental—we build on past knowledge more often than we displace it. Thus, the first pillar of science—a communal collection of knowledge—requires mechanisms for preserving and disseminating knowledge within the scientific community. By far the most important mechanism in use today is the scientific publication. (A second important mechanism is the science textbook and classroom, which will not be addressed here.) While there are many forms of scientific publication, the two most common are the conference presentation (with or without some non-peer-reviewed written text), and the peer-reviewed journal paper (both in print and online).
Since virtually all scientific advances build on past knowledge, it is critical that the new work be placed in the proper context with respect to the past work upon which it builds. The primary mechanism for this is the citation (or reference). Within a scientific paper, references are placed to other works, creating points of contact with the communal collection of scientific literature in order to fit the new work into the web of knowledge. But given the skeptical attitude that is also a part of science, citations are also used to help readers verify the quality of the new work and assess the strength of its conclusions.
A citation is, by definition, a reference to a source of information or data. Things that can be cited include journal articles, conference proceedings, books, student theses, newspapers, nonprint sources (such as film or other recorded media), websites or other online resources, computer materials (such as a published CD-ROM of data or a piece of software), and personal communications. The citation should be located in the text in such a way that it is clear what material requires the citation. Often this is at the end of a sentence, but sometimes it must be put in the middle of the sentence to enhance clarity. Obviously, citations must supply sufficient detail so that the referenced material can be found and uniquely identified. As such, every journal establishes a specific format for citations that must be followed. The JM3 format for references can be found at http://spie.org/x85020.xml.
Though simple in concept, citations in a scientific paper serve many goals. The five most important goals are:
• Provide sufficient context of the work to allow for critical analysis of the work by others, and thus to enable the readers to gauge for themselves whether the author’s conclusions are justified;
• Give the reader sources of background and related material so that the current work can be understood by the target audience (thus creating a web of science);
• Establish credibility with the reader (e.g., the author knows the field, has done his/her homework, etc.) and/or inform the reader that the paper belongs within a specific school of thought;
• Provide examples of alternate ideas, data, or conclusions to compare and contrast with this work;
• Acknowledge and give credit to sources relied upon for this work (i.e., acknowledge the use of another’s ideas or data), thus upholding intellectual honesty.
Of these five goals, the most commonly mentioned is to give credit to others (the so-called normative theory of citations2), and thus demarcate what credit is due the new work. Let’s face it, scientists can have big egos. We’re frequently motivated by the desire for peer recognition. Thus, we try to carefully stake a claim to new ideas or data in our paper, knowing full well that others will be checking to make sure we don’t claim too much. Even so, there is no pretense that a list of references will provide a complete list of influences; such a list would be excessive in even the simplest of cases.
While important, the “give credit” purpose of citations is, in my opinion, less compelling than the other goals. I view citations, like all aspects of scientific writing, from a simple perspective: what best serves the needs of the reader? Thus, the primary goal of citations should be to help the reader gain the most from the paper. Imagine your paper being read by a graduate student or postdoc: smart, but new to the field. If she reads all of the citations, will she have enough background to understand your work? Will any of the references be unneeded or redundant (and thus a waste of the reader’s time to look up)? Chances are very good that this simple test will be sufficient to decide on most references: will adding this reference here make the paper more valuable to the reader, or less?
The Literature Search
A new research project almost always begins with a literature search—or at least it should. The goal of the literature search is to evaluate the state of our communal knowledge on a topic before embarking on a quest of adding to that knowledge. Since science is either about confirming or refuting existing knowledge or developing new knowledge, a thorough understanding of the current state of communal knowledge is essential. Additionally, this literature search will form a foundation for all of the five goals of citations. Note that a literature search is not about finding relevant papers, it is about reading relevant papers.
Unfortunately, literature searches are rarely done as well as they should be. Here are a few hints to improve literature searches:
• Do the literature search before performing the research, and certainly before writing the paper. Doing a literature search at the end often generates spurious citations (a problem that will be discussed below).
• The most promising next papers to read are often those referenced in the relevant papers you have already found. (This is another reason why good citations are important—their influence tends to multiply.)
• Look in fields outside your discipline (this often means looking for different search keywords, which one recursively discovers when reading the literature outside of one’s discipline).
• While your memory of what previous papers are worth citing is a good start, no one ever knows the full scope of the literature in even the smallest of niche fields. Don’t rely on your memory alone.
• When finishing up the manuscript, look for recent publications on the subject. Often, other researches are working on similar topics and may have published papers that should be read to ensure your manuscript captures the latest communal knowledge in the field.
Starting a literature search always leads to a difficult question: How do you know when to stop? There will always be important papers that you never find. This is the nature of modern science. Knowing when to quit (or pause) the literature search and begin the new work is a matter of judgment and experience.
Verify, Verify, Verify
One of the most pervasive problems with citations is that they are frequently incomplete or inaccurate. It is the job of the authors to verify the accuracy of the references. Editors, copy editors, and reviewers are not responsible for reference accuracy and are not expected to check references for accuracy. And while copy editors try to flag incomplete or improperly formatted references, it is the authors that ultimately must fix the errors found. Why not do the work up front to ensure that the references are complete, accurate, and properly formatted? It will only save time and effort in the end, and indicate to the editors and reviewers that you care enough to pay attention to these important details.
Alas, far too few authors take this advice seriously. Several studies have found that between 34 and 67% of references in a variety of medical and biomedical journals contained errors.3* These errors can be broken down into major and minor errors. A major error means that the article could not be found given the information in the citation. One study found that major errors occur in 7% of the citations from one class of medical journals.4 Minor errors include punctuation or spelling mistakes, mistakes in the article titles, mistakes in the name and initials of the author(s), and citation style mistakes. These errors serve as irritants to the reader—they can still find the article, but have to put more effort into it to do so.
It is probably obvious that the main cause of errors in citations is simple sloppiness on the part of the author. There is another problem, however, that may also be at work: copying citations from other papers. In other words, some authors commit a cardinal sin of citations and add a reference without ever having read that paper. Copying citations from other papers without actually looking up and reading that paper can result in a propagation of errors that are never corrected5 (kind of like the telephone game that my daughters like to play). A slightly less egregious form is the abstract citation: citing a paper after reading only the abstract. Both types of unread cites should be avoided: cite only papers you have read.
Other Problems with Citations
There are other reasons why a specific reference does not fulfill the goals set out above and thus does not benefit the reader.
Spurious citations: citations that are not needed but are included anyway. Sometimes these citations are added at the last minute, after the paper is written, to give the impression that a literature search and proper citation work have been done. Often they include redundant citations, where the extra citations do not add any value beyond the first one. An example given by Brian Thompson6 was “related work on the technique has been carried out by numerous researchers.1-101” The problem is obvious: an interested reader must wade through far too much literature to get the needed background. Sometimes spurious cites are meant to give an impression of erudition by citing an obscure, historical reference (if the referenced work is in a foreign language, all the better).7 In all such cases, simply asking the question “If the reader looks up this reference, will it be time well spent?” will be enough to decide if that reference is spurious.
Biased citations: references added (or omitted) for reasons other than meeting the five goals of citations. Biases include overciting of friends’ or colleagues’ work, omitting cites to the work of rivals, and gratuitous citations in an attempt to curry favor with a boss or potential referee.
Self-cites: citations to one’s own work. There is nothing wrong with self-citations, per se. After all, the work represented in a single paper is often just the latest result of a larger ongoing project. As such, citations to one’s earlier work are often perfectly appropriate. Self-cites are a problem when they are either spurious or biased.8 Knowing as we do the tendencies of many scientists toward self-promotion, one fears that self-cites may be designed to boost the recognition of the author rather than increase the value of the paper to the reader.
Excluding contrary evidence: a form of biased citations where citations to prior work whose conclusions or data contradict the current work are omitted. Since one of the five goals of citations is to explicitly contrast the new work with prior work containing conflicting data or conclusions, avoiding such conflict (for whatever reason) does not serve the interest of science.
In the end, authors must find a balance between too many and too few citations. The literature base even on very narrow topics is often vast, and it can be difficult to pick a small subset to cite.
To do a good job of providing citations in a scientific publication, one must keep in mind the multiple goals of proper citing. But like other aspects of good science writing, a simple theme has emerged: make the paper reader-centric, not author-centric. While it is common to choose citations that make the paper more valuable to the author (by demarcating what is novel, for example), good citations make the paper more valuable to the reader. Unfortunately, doing a good job of citing requires more work from the authors. But careful citing is worth the effort if your goal is a quality scientific publication.
R. K. Merton, The Sociology of Science: Theoretical and Empirical Investigations, University of Chicago Press, Chicago, IL (1973).Google Scholar
M. H. MacRobertsB. R. MacRoberts, “Problems of citation analysis: a critical review,” J. Am. Soc. Inform. Sci. 40(5), 342–349 (1989).AISJB61097-4571http://dx.doi.org/10.1002/(ISSN)1097-4571Google Scholar
A. E. MohammadD. M. Laskin, “Citation accuracy in the oral and maxillofacial surgery literature,” J. Oral Maxillofac. Surg. 66(1), 3–6 (2008).JOSUA90278-2391http://dx.doi.org/10.1016/j.joms.2007.06.682Google Scholar
M. V. SimkinV. P. Roychowdhury, “Read before you cite!,” Complex Systems 14(3), 269–274 (2003).CPSYEN0891-2513Google Scholar