Exploring biases and potential effects of S&T indicators in peripheral spaces

Ismael Rafols
Instituto de Gestión de la Innovación y del Conocimiento, INGENIO (CSIC-UPV)
Tuesday, 28 July 2015 - 12:00

Exploring biases and potential effects of S&T indicators in peripheral spaces


Jordi Molas-Gallart1*, Ismael Rafols1,2+, Diego Chavarro2 and Richard Woolley1

1Ingenio (CSIC-UPV), Universitat Politècnica de València, València, Spain

2 SPRU (Science Policy Research Unit), University of Sussex, Brighton, England,

* jormoga@ingenio.upv.es,


This paper aims to explore the problems that emerge when S&T indicators are used in peripheral contexts, that is, in geographical or social spaces that are somehow marginal to (or marginalised by) the centres of scientific activity. In these situations evaluators and decision-makers are likely to use indicators that were designed to reflect variables relevant in the dominant social and geographical contexts --i.e. in the hegemonic countries, languages, gender, disciplines, etc.--, but that are usually not adequate in peripheral contexts.

We will examine various dimensions of periphery. First, the geographical: e.g. global south vs. global north, regions vs. metropolises (Aguado et al. 2014). Second, the social group dimension: women, the disenfranchised, the poor, or perhaps the elderly have social needs that are different from those of richer or more powerful groups --and the problems affecting the former tend be less researched than those of the later (Stirling, 2014). Third, the cognitive dimension: areas of research, such as epidemiology or surgery, that capture less attention in terms of publications or citations (and resources) than the more prestigious disciplines, such as molecular biology (van Eck et al, 2013).

This study investigates the mechanisms by which performance indicators tend to be biased against  peripheral spaces. This would include for example, bias in language (van Leeuwen et al. 2011), or disciplinary/topic coverage in conventional databases (Martin et al., 2010). An interesting issue to consider is how the overlap across peripheries, i.e. how bias in language coverage has an effect on bias in disciplines or topics covered (Archambault et al., 2006; Piñeiro and Hicks, 2015).

We discuss how these biases may have a tendency to suppress scientific diversity and shift research towards a higher degree of homogeneity (Rafols et al., 2012). We discuss how the "objectification" of excellence by means of indicators may support the diffusion of mainstream modes of research at the expense of critical or unorthodox modes.


Aguado-López, E., Becerril-García, A., Arriola, M. L., & Martínez-Domínguez, N. D. (2014). Iberoamérica en la ciencia de corriente principal (Thomson Reuters / Scopus): Una región fragmentada. Interciencia, 39(8), 570-579.

Archambault, É., Vignola-Gagne, É., Côté, G., Lariviere, V., & Gingras, Y. (2006). Benchmarking scientific output in the social sciences and humanities: The limits of existing databases. Scientometrics, 68(3), 329-342.

Martin, B. R., Tang, P., Morgan, M., & al. (2010). Towards a Bibliometric Database for the Social Sciences and Humanities – A European Scoping Project (A report for DFG, ESRC, AHRC, NWO, ANR and ESF). Brighton, UK: SPRU.

Piñeiro, C. L., & Hicks, D. (2015). Reception of Spanish sociology by domestic and foreign audiences differs and has consequences for evaluation. Research Evaluation, 24(1), 78-89.

Rafols, I., Leydesdorff, L., O’Hare, A., Nightingale, P., Stirling, A., 2012. How journal rankings can suppress interdisciplinarity. The case of innovation studies and business and management. Research  Policy 41, 1262–1282.

Stirling, Andy. Towards Innovation Democracy? Participation, Responsibility and Precaution in Innovation Governance. No. 2014-24. SPRU-Science and Technology Policy Research, University of Sussex, 2014.

Van Eck, N. J., Waltman, L., van Raan, A. F. J., Klautz, R. J. M., & Peul, W. C. (2013). Citation Analysis May Severely Underestimate the Impact of Clinical Research as Compared to Basic Research. PLoS ONE, 8(4), e62395. doi:10.1371/journal.pone.0062395


Ciudad Politécnica de la Innovación
Edificio 8E, Acceso J, Planta 4ª (Sala Descubre. Cubo Rojo)
Universidad Politécnica de Valencia | Camino de Vera s/n

Short CV: 

Soy un analista de políticas de ciencia y tecnologia en Ingenio (CSIC-UPV), investigador visitante a SPRU (Univ. Sussex)(donde estuve del 2004 al 2012), y científico associado al Observatoire des Sciences et des Téchnique (OST-HCERES, Paris). Trabajo sobre evaluación de ciencia y tecnología. Estoy interesado en la pluralización y democratización de la investigación. Con este objectivo, desarrollo métodos para mapear investigaciones interdisciplinares, intercambios de conocimiento y contribuciones sociales de la ciencia. En este momento estoy creando instrumentos para la visualización del tipo de investigación (los repertorios) existente sobre grandes retos como la fiebre aviar o la obesidad. También analizo el tipo de investigación desarrollado por grandes organitzaciones públicas (com universidades) o privadas (comfarmacèuticas).

Soy miembro de los consejos editoriales de ScientometricsJournal of InformetricsPeerJ Computer Science, y Journal of Policy and Complex Systems, y participos en los comités de evaluación de conferencias como Atlanta Science PolicyENIDor Eu-SPRI.

Google Scholar muestra mis publicaciones en acceso abierto (verde). Currículum Vitae (en inglès).