“Science does not exist until it is published”.1 A scientific journal first appeared in 1665 and the refereeing of manuscripts began in 1752.2,3 Thousands of scientific journals are now available. The ranking of journals is difficult. The impact factor has emerged as an index of quality and prestige, and is used increasingly for ranking and evaluating journals.4–9 Detailed lists of journal citations and impact factors are produced and made available by the Institute for Scientific Information (ISI). A listing of all the impact factors is included in the Journal Citation Report (JCR).10 The impact factor in a particular year is the ratio of the number of citations in the current year to articles published in the journal in the two preceding years divided by the number of citable items published in the same two years.10–12
Self-citation and the citation density of a journal, which is the number of references listed, may affect its impact factor.13,14 In this study, we have investigated the self-citation and citation density of seven orthopaedic journals and their possible effect on the impact factor.
Self-citation during 2002 and 2003 was investigated in the American volume of the Journal of Bone and Joint Surgery (JBJS [Am]), the British volume of the Journal of Bone and Joint Surgery (JBJS [Br]), Clinical Orthopaedics and Related Research (CORR), Acta Orthopedica Scandinavica (Acta Orthop Scand), International Orthopaedics (Int Orthop), Orthopaedic Clinics of North America (OCNA) and the Archives of Orthopaedics and Trauma Surgery (Arch Orthop Trauma Surg). Abstracts, supplements, letters, editorials and case reports were excluded from the study.
We calculated the self-citation rate as follows:
For example, the JBJS [Am] made references to 5676 articles, including 2304 of its own. This gave a self-citation rate of 2304/5676 = 40.59% (Table I⇓). The self-citation rates of the seven orthopaedic journals in the year 2002 and 2003 were correlated with their impact factors published in the year 2004 taken from the Science Citation Index12 of the JCR (Fig. 1⇓).
We also calculated the self-cited rate for the seven orthopaedic journals as follows:
For example, during the two-year period JBJS [Am] was cited 8053 times in the seven journals, including the 2304 times that it was cited by itself. Its self-cited rate was 2304/8053 = 28.61%. Since we only included the seven orthopaedic journals, the calculation of the self-cited rate differs from the standard definition derived from the ISI. The latter defines the self-cited rate as the ratio of the journal’s self-citation to the number of times which it is cited by all other journals, including itself.10
We also calculated the citation density for the seven orthopaedic journals (Table I⇑) as follows:
The citation density of JBJS [Am] was 14 696/451 = 32.58%.
This was performed using SPSS for Windows 11.0 (SPSS Inc, Chicago, Illinois). The citations attributed to each journal by the other six journals were compared among the seven journals using the chi-squared test. To identify the contribution of different cells to the significance of this chi-squared test, we used adjusted standardised residuals. These followed the t- distribution. For a standardised residual > 1.96, Spearman’s rank (rs) correlation coefficient was used to assess the correlation between the self-citation rates of the journals and their impact factors. Adjusted residuals can be interpreted like z-scores. If they are > 1.96, they are significantly different from zero at the 95% confidence level.
We assessed 2352 articles in the seven orthopaedic journals between 2002 and 2003. There were 24 378 citations.
Table I⇑ shows that JBJS [Am] and CORR had the highest self-citation rate (40%) while OCNA, Int Orthop and Arch Orthop Trauma Surg had the lowest. There was a significant correlation between the self-citation rate and their impact factor for the seven journals (r = 0.786; p = 0.0001; Fig. 1⇑).
Individual comparison (Table II⇓) showed that JBJS [Am] favoured itself, had no significant effect on OCNA and CORR, but had a significant negative effect on JBJS [Br], Acta Orthop Scand, Int Orthop and Arch Orthop Trauma Surg. JBJS [Br] favoured itself and Acta Orthop Scand and had no significant effect on Int Orthop and Arch Orthop Trauma Surg, but had a negative effect on the other three American journals. CORR favoured itself and OCNA; it had negative effect on the other five journals. OCNA favoured itself and JBJS [Am] with a negative effect on the other five journals. Acta Orthop Scand favoured itself, JBJS [Br] and Arch Orthop Trauma Surg. It was indifferent to Int Orthop and had a negative effect on the rest. Int Orthop favoured itself and JBJS [Br], was indifferent to CORR and Acta Orthop Scand with a negative effect on the other three journals. Arch Orthop Trauma Surg favoured itself, Acta Orthop Scand and Int Orthop, had a negative effect on JBJS [Am] and was indifferent to the other three journals.
The self-cited rates are given in Table I⇑. The citations each journal gave to the other journals, including itself, and the citations each journal received from the other journals differed significantly among the seven (chi-squared test = 1362, df = 36, and p < 0.0001; Table II⇑).
There was a significant correlation between the citation density and the impact factor for the seven orthopaedic journals (r = 0.613).
We did not include subspecialist journals such as Arthroscopy, Shoulder and Elbow, Knee, and Arthroplasty, since they are likely to have more self-citation because of subject subspecialty. We selected three orthopaedic journals from the USA and four from Europe, since the United States and Europe dominate medical publishing.
Each journal had a significantly higher self-citation rate compared with the self-cited rates which they gave to the other journals. Those with a self-citation rate above 20% are defined as having high self-citation rates.10 JBJS [Am], JBJS [Br] and CORR all had a self-citation rate above 20% and had a high impact factor compared with the other four orthopaedic journals.
JBJS [Am] favoured itself more than the other journals. It had no positive effect on any of the other journals, including those from the USA. JBJS [Br], CORR, and Acta Orthop Scand also had similar self-citation rates, which were relatively close to their self-cited rates. Thus, the rate of citation of these journals by all journals, including themselves, did not appear to be affected positively by their self-citation.
European journals showed mutual support for other European journals. American scientists are prone to citing each other’s work.15 A similar study of journals in anaesthesia13 found that North American journals cited the North American literature. This was less noticeable in the European journals of anaesthesia. It was also found that the self-citation rate influenced the impact factor of the journals of anaesthesia. In otolaryngology there was no significant correlation between self-citation rates and the impact factor.16
Citation density is considered to influence the impact factor of a journal.14 Few journals set limits on the number of references which an author is allowed to include in a manuscript submitted for publication. Even closely-related journals in the same specialty differ in the citation density, from 18 to 60 in our study (Table I⇑). The journals with a larger citation density also tended to have larger impact factors.14,15
The impact factor of a journal represents the citation frequency of the average published article and not a specific paper.14 It gives a quantitative measure of the effect which a journal has in its area of specialisation. It should not be used to evaluate and compare the productivity and importance of the work of individual scientists. There is a poor correlation between the impact factor of a journal in which an article is published and the number of future citations to that article.16
There are many limitations and biases in calculating the impact factor of a journal.17 The type of the journal, its size, domain concerned, language of publication and choice of the manuscript published such as topical subjects or review articles, influence the citation rate of an article and therefore, the impact factor of the journal.13,16,18 A relatively high self-citation rate can be due to several factors. Authors may prefer to submit their work on a particular topic to a journal which has previously published relevant work in that area. Therefore more references in the author’s article will be derived from articles in that particular journal.
The significant correlation between the self-citation rate and the impact factors of the seven orthopaedic journals indicates a significant influence of self-citation on their impact factors. If the self-citation rate significantly supersedes the self-cited rate then the journal’s impact factor may be favoured by its own self-citations.
A high self-citation rate and citation density may significantly influence the impact factor of a journal. This could be overcome by an article-by-article citation count and calculation of the impact factor after elimination of self-citing, correction by a factor for self-citation and setting the limit for the citation density.
- © 2006 British Editorial Society of Bone and Joint Surgery