Abstract
This paper examines a number of the criticisms that citation analysis has been subjected to over the years. It is argued that many of these criticisms have been based on only limited examinations of data in particular contexts and it remains unclear how broadly applicable these problems are to research conducted at different levels of analysis, in specific field, and among various national data sets. Relevant evidence is provided from analysis of Australian and international data. Citation analysis is likely to be most reliable when data is aggregated and at the highly-cited end of the distribution. It is possible to make valid inferences about individual cases, although considerable caution should be used. Bibliometric measures should be viewed as a useful supplement to other research evaluation measures rather than as a replacement.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bourke, P., L. Butler (1998) Institutions and the map of science: matching university department and fields of research,Research Policy, 26:711–718.
Bourke, P., L. Butler (1996) Publication types, citation rates, and evaluation,Scientometrics, 37:473–494.
Bourke, P., L. Butler (1993),A crisis for Australian science, Performance Indicators Project, Monograph Series No. I, Canberra: The Australian National University.
Carpenter, M., F. Narin (1981) The adequacy of the Science Citation Index (SCI) as an indicator of International Scientific Activity,Journal of the American Society for Information Science, 32:430–439.
Cole, J., S. Cole (1973)Social stratification in science, Chicago: University of Chicago Press.
Cole, S., T. Phelan (1999)The scientific productivity of nations, Minerva 37:1–23.
Crane, D. (1972)Invisible Colleges: Diffusion of Knowledge in Scientific Communities. Chicago: University of Chicago Press.
Davies, D. J., J. E. Moran, J. F. Niall, G. B. Ryan (1982) Segmental necrotizing glomerulonephritis with anti-neutrophil antibody-possible arbovirus aetiology?British Medical Journal, 285:606.
Garfield, E. (1998) Random thoughts on citationology. Its theory and practice.Scientometrics, 43:69–76.
Gibbs, W. W. (1995). Lost Science in the Third World,Scientific American. August:76–83.
Holton, G. (1978) Can Science be measured, inToward a Metric of Science Y. Elkana et al. (Eds.) New York: Wiley.
Kostoff, R. N. (1998) The use and misuse of citation analysis in research evaluation,Scientometrics, 43:27–43.
Lindsey, D. (1989) Using citation counts as a measure of quality in science: Measuring what's measurable rather than what's valid.Scientometrics, 15:189–203.
MacRoberts, M. H., B. R. MacRoberts (1996) Problems of citation analysis,Scientometrics, 36:435–444.
MacRoberts, M. H., B. R. MacRoberts (1989) Problems of citation analysis: a critical review,Journal of the American Society for Information Science, 40:342–349.
Martin, B. R. (1996) The use of multiple indicators in the assessment of basic research,Scientometrics 36:343–362.
Martin, B. R. (1994) British science in the 1980s: has the relative decline continued?Scientometrics, 29:27–56.
Martin, B. R., J. Irvine (1983) Assessing basic research: some partial indicators of scientific progress in radio astronomy,Research Policy, 12:61–90.
Phelan, T. (1998)Is Australian Educational Research Worthwhile? Unpublished manuscript. The Australian National University.
Van Raan, A. F. J. (1996) Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises.Scientometrics, 36:397–420.
Wade, N. (1975) Citation analysis: A new tool for science administrators,Science, 188:429–432.
Zuckerman, H. A. (1977)The Scientific Elite, New York: Free Press.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Phelan, T.J. A compendium of issues for citation analysis. Scientometrics 45, 117–136 (1999). https://doi.org/10.1007/BF02458472
Received:
Issue date:
DOI: https://doi.org/10.1007/BF02458472
