Study of the Most Valuable Researcher: Bibliometric Indicators
and Collaboration Networks*
Camilo Peña Ramírez**
https://orcid.org/0000-0002-1535-8510
Universidad Central de Chile, Chile
Leonardo Concha*
Universidad del Bío-Bío, Chile
Eric Forcael*
http://orcid.org/0000-0002-3036-4329
Universidad del Bío-Bío, Chile
Gonzalo Garcés*
https://orcid.org/0000-0002-1359-4835
Universidad del Bío-Bío, Chile
Recibido: 17 de junio del 2021 / Aprobado: 27 de julio del 2021
doi: https://doi.org/10.26439/ing.ind2021.n41.5208
ABSTRACT: This work seeks to find the Most Valuable Researcher (MVR) within the academics of Faculties of Engineering and Business of a University in Chile, applying bibliometric indicators and collaboration networks. The methodology consisted in reviewing the literature referring to similar bibliometric studies from open databases, such as SciELO and Google Scholar. As a result of the study, a model was proposed based on the main bibliometric indicators used, with it was possible to filter the researchers from both faculties and establish a ranking with those academics with the best results and the current situation facing the research in each unit. This ranking indicates the standard that the most valuable researchers have, identifying that the variable “collaborative networks” has a direct relationship with the productivity of researchers and, also, the existence of correlations with indicators of network grade, co-authorship, and research area. This work seeks to deliver recommendations on the quantity and quality of scientific production within the University. Future research should include other databases and expand the scope by region, country, and area of expertise, and consider other factors such as the age of the researcher, forms of citation, and characteristics by area of knowledge, as well as deepen the concept of MVR, and its virtuous effect on the productivity of an academic unit.
KEYWORDS: most valuable researcher / bibliometric indicator / collaborative networks
/ scientific production / WoS, h index
Estudio del investigador más valioso: indicadores bibliométricos y redes de colaboración
RESUMEN: Este trabajo busca encontrar al Investigador Más Valioso (IMV) entre los académicos de las Facultades de Ingeniería y Negocios de una Universidad en Chile, aplicando indicadores bibliométricos y redes de colaboración. La metodología consistió en una revisión de la literatura referida a estudios bibliométricos similares de bases de datos abiertas, como SciELO y Google Scholar. Como resultado del estudio, se propuso un modelo basado en los principales indicadores bibliométricos utilizados, con lo cual se pudo filtrar a los investigadores de ambas facultades y establecer un ranking con aquellos académicos con mejores resultados y la situación actual que enfrenta la investigación en cada unidad. A partir de este ranking, se indica el estándar que tienen los investigadores más valiosos, identificando que la variable “redes de colaboración” tiene una relación directa con la productividad de los investigadores, así como la existencia de correlaciones con los indicadores de grado de red, coautoría y área de investigación. Con esto, se busca que este trabajo sirva para entregar recomendaciones sobre la cantidad y calidad de la producción científica dentro de la Universidad. Futuras investigaciones deberían incluir otras bases de datos y ampliar el alcance por regiones, países y áreas de conocimiento y considerar también otros factores como la edad del investigador, formas de citación y características por área de conocimiento, así como profundizar en el concepto de IMV y su efecto virtuoso en la productividad de una unidad académica.
PALABRAS CLAVE: investigador más valioso / indicador bibliométrico / redes
de colaboración / producción científica / WoS / índice h
1. Introduction
This work seeks to determine the main factors that intervene in the quality and quantity of scientific research at a university level. The search for the best producers of scientific material was carried out within the Faculties of Engineering and Business at the main campus of a regional university in Chile.
The study’s main objective is to determine the most valuable researcher (MVR) using bibliometric tools. By definition, bibliometrics focuses essentially on calculating and analyzing the values of what is quantifiable in the production and consumption of scientific information (Roig-Tierno et al., 2017). On the other hand, Okubo (1997) proposed bibliometry as the discipline that measures and analyzes the production of science in the form of articles, publications, citations, patents, and other more complex derived indicators that make it possible to determine each of the factors that intervene in the scientific evaluation and the performance of researchers.
The study considers three areas in applying these indicators: the first is the bibliometric data of Web of Science, and the second is the collaborative networks of the authors in their publications. As a complement, it extracts the indicators with what Google Scholar works about the authors. In this way, a comparison is made between data delivered by the Web of Science and Google Scholar.
2. Theoretical framework
2.1 Bibliometrics
Bibliometrics is an emerging and frontier research field of Library and Information Science (LIS), which has significantly developed in recent decades (White & McCain, 1998).
As Alan Pritchard (1969) proposed, bibliometrics applies mathematics and statistical methods to analyze the course of written communication and the course of a discipline. It applies a quantitative treatment to the properties of written speech and its typical behaviors. Other terms such as Scientology, Reporting, Librarianship, Webometrics are also frequently used; each refers to metric studies applied to specific phenomena or objects (Chaparro & Rojas-Galeano, 2021; Roldan et al., 2019; Sengupta, 1992).
Later, Okubo (1997) defines bibliometrics as the discipline that measures and analyzes the production of science in the form of articles, publications, citations, patents and other more complex derived indicators.
Among the first antecedents of bibliometric studies found in the literature, the works of Cole and Eales (1917) are usually quoted. They made a statistical analysis of comparative anatomy publications between 1550 and 1860, distributed by country and the divisions of the animal kingdom. Similarly, Hulme (1923), librarian of the British Patent Office, presented a statistical study of the History of Science, and Gross & Gross (1927) on references included in chemistry journals indexed in the Journal of the American Chemical Society.
2.2 Bibliometric Methods
Bibliometric methods are based on the premise that the number of publications shows the scientific productivity of an individual or a research group at a local, regional, national or international level (Chaparro & Rojas-Galeano, 2021; Patra et al., 2006; Roldan et al., 2019). In this respect, any methodological approach to bibliometrics relies on recognizing a database (mainly contained in the Science Citation Index, Web of Science, Social Sciences Citation Index). A set of selected indicators is classified from the databases to identify their frequency and distribution for a particular discipline or scientific area (Zupic & Čater, 2015).
One of the best-known databases worldwide is the Web of Science (WoS), a product of the ISI Web of Knowledge package, which currently belongs to the company Clarivate Analytics. What ISI does through WoS is basically to integrate an extensive database of published articles identified from the world’s leading academic journals, where most of the indicators used to interpret the information are proposed by the international academic community (Jacsó, 2008).
Numerous studies show bibliometric methodologies applied to disciplines, knowledge areas and countries. Canada is an example of a country that has continuously worked with bibliometric methods to measure the results of their scientific and technological work (Gringas, 1996).
2.3 Bibliometric Indicators
Bibliometric indicators are statistical data derived from scientific publications. They are based on the crucial role of publications in disseminating new knowledge, a role assumed at all levels of the scientific process. These indicators are valid in research that leads to scientific publications, common in the most fundamental scientific areas (King, 1987; Rinia et al., 1998).
The main bibliometric indicators can be grouped into two basic categories: (a) quantitative indicators of scientific activity (number of publications) and (b) impact indicators, based on the number of citations obtained from the works, that characterize the importance of these productions based on the recognition granted by other researchers (Bordons, 1999).
That said, table 1 shows the main indicators used in the study.
Table 1
Summary of indicators
Indicator |
Name |
Description |
|
Bibliometrics WoS |
Citations |
Total citations received |
Correspond to the total number of citations that the author has in the WoS database. |
Total publications |
Delivers the number of publications made in WoS. |
||
Citation average |
Delivers the average number of citations received per article. |
||
h index |
Impact- h |
It quantifies the productivity of the authors according to the number of citations received and the total number of publications (WoS). |
|
Productivity |
Annual Productivity |
It indicates the average of the researcher’s annual publications. |
|
Lotka Productivity |
It classifies the author in two grades, elite and casual. Elite corresponds to the most productive and casual to the least productive. |
||
Impact |
Impact Factor |
It indicates the impact caused by the publication in the journal. It takes values between 1 and 4, with 1 being the best factor and 4 the lowest. |
|
Google Scholar |
h index |
Impact h |
It quantifies the productivity of the authors according to the number of citations received and the total of publications based on the information provided by Google Scholar. |
Citations |
Total citations received |
It indicates the total number of citations received in their publications according to Google Scholar. |
|
Networks |
Grade |
Country Network Grade Indicator |
The number of countries to which the author is related. |
Co-authority |
Co-authority Indicator |
The network of co-authors with whom it conducts the research |
|
Network’s size |
External network’s size |
The number of researchers to whom the author is related. |
|
Area |
Area of research |
The number of research areas with which the authors are related in their publications. |
Own elaboration based on theoretical framework
3. Methodological framework
Figure 1 presents the research methodology used to identify the most valuable investigator (MVR) standard.
Figure 1. Proposed methodology
Own elaboration
We have the methodology to follow and the indicators to be used; however, in addition to carrying out the descriptive statistical analysis of the data, we perform a multiple regression analysis with the selected indicators to enable a comparison between the results of the descriptive analysis of the multiple regression analysis. Figure 2 shows the Most Valuable Researcher (MVR) model:
To select the sample data, publications made by Faculties of Engineering and Business, both belonging to a regional University in Chile, were selected. These were selected to make a comparison and establish differences or similarities between the indicators obtained.
We applied bibliometric indicators to data obtained from the WoS and, afterward, compared some of them with indicators obtained from Google Scholar’s researcher profile and, finally, applied productivity filters. The group that stands out is selected within the publications (“elite”) according to the Lotka indicator, to then apply the selected network indicators. It should be noted that the study did not consider the variables “time in the institution” or “academic load of researchers” because they did not appear in the literature consulted. Applying these variables would increase the complexity of the analysis.
Figure 2. Most Valuable Researcher (MVR) model
Own elaboration
To achieve the main objective, we filtered the selection by organization (OG = University -name-) in the WoS and obtained 1792 publications. Subsequently, the search was refined considering ten years of publications between 2008 and 2018, thus reducing data to 1502 publications. Here we found a big problem, since the WoS does not have a faculty filter, it was necessary to verify each of the publications of these ten years and save, with the help of Microsoft Excel, only the data valid for the study. Finally, the database was limited to a total of 281 exclusive publications of both faculties.
The data extracted from the WoS included: title of the research, year and month of publication, authors and co-authors, institution and area of each author of the publications, language, the quartile of publication, number of citations received, country and city of researchers, as well as the h index and research areas with which the researchers are associated. A unique five-digit ID replaced the name of each author and co-author to avoid confidentiality problems.
Having all the necessary data, we used indicators for each area of study and obtained bibliometric indicators. Thus it is possible to obtain indicators of citations such as the total number of citations received, the total of publications by author, and an average of citations received by each article. The h indicator, which quantifies productivity, is obtained directly from the WoS, so it does not need to be calculated. The annual productivity of researchers can be obtained through dynamic tables with the productivity indicators and the Lotka indicator. Finally, we considered the impact category of the research and classified the authors according to the impact of the journals in which they publish.
The second step was to compare indicators of citations and the h index obtained from the database, with the data extracted from the profiles of each researcher (the “elite”) in Google Scholar, this in order to determine if the ranking obtained only for ISI publications of WoS is similar to that obtained in Google Scholar under all publications of the authors (not only ISI).
Finally, we applied indicators in collaborative networks, among which the following stand out: (1) grade indicator in countries (the number of countries with which the author is related); (2) size of the network (the number of researchers with whom the authors are related); (3) research area in which the authors collaborate (in this indicator the sub-areas were not considered because the study applied to only one institution and with a small number of authors); and (4) the number of co-authors with whom they work.
4. Results
After using the filters, the 281 articles selected from WoS were saved in a Microsoft Excel 2016 database to enable the necessary calculations. The results were 72 articles belonging to the Faculty of Business and 209 to the Faculty of Engineering, that is 25,62 %
and 74,38 %, respectively.
4.1 Publications per year
Publications of both faculties during the years under study reflect an increase in the number of articles published by the Faculty of Engineering in WoS, reaching 33 published pieces of research in 2016, an annual average of 19 publications, and a standard deviation of 9,6. On the other hand, the Faculty of Business has fewer publications than the Faculty of Engineering in each year of comparison; it achieved its highest number of published articles in 2017, with 14 articles; its average of publications is six per year, with a standard deviation of 3,41.
4.2 Publication Language
The language of an article seriously conditions the number of citations and the impact factor of a journal. Ninety seven percent of the scientific journals that appear on the Web of Science (WoS) are written in English (Franco-López et al., 2016). The difference in the penetration of articles in English is abysmal in comparison to other languages. That is why English is considered the international language for scientific research, improving visibility and increasing the probability of receiving citations (Cargill and Burgess, 2017).
4.3 Authorship of publications
The database allowed to retrieve the number of authors participating in the publications. Considering researchers from the University and other institutions, the total amount was 520; 116 belong to the Faculty of Engineering, 48 to the Faculty of Business, and 356 to other institutions.
4.4 Lotka Indicator
With this indicator, it was possible to filter the number of authors for the study, since many of them present a low number of publications; 67 authors of the Faculty of Engineering have only one publication, and a single researcher has 30 investigations, which produces a significant difference between the authors.
4.5 Application of Lotka to the database
Applying the method delivered by Lotka reduced the number of researchers. For the Faculty of Engineering, the number of researchers with which it will be necessary to work is ten, that is to say, the ten researchers who have the highest number of publications. Just six in the case of Business School.
4.6 Index of citations for authors, Faculty of Engineering
The ten authors of the Faculty of Engineering received at least 57 citations for their publications, with author 78120 being the most cited, reaching 124 citations, followed by author 72199 with 102, and author 72149 with 64. The two authors who received the most citations also have the most publications with 21 and 30, respectively. When the average of citations per author was calculated, the group average was 4,62 citations for each publication they participated in, with a standard deviation of 1,65. However, this result does not reflect what happens with each researcher. Author 72149 received only 2,8 citations per article, while author 72174 received an average of 7,2 citations per article.
The analyzsis results showed that authors with higher productivity are not necessarily those with the highest number of publications; instead, the critical factor is the number of years that it takes to produce. The author with the highest productivity was 78120 with an average of 4,2 publications per year, while the author 72199, with 30 publications, only achieved a productivity index of 2,7 publications per year. On the other hand, the worse index obtained was by author 78101, who failed to achieve at least one publication per year.
4.7 Index h
The results in table 2 show, for each author, that —in their great majority— they have similar h indicators, with an index of 5, which means that there are five publications in which the authors received five citations in each one of them. Several types of research approved the use of this indicator, making a ranking of researchers based on the h index (Braun et al., 2005; Mitra, 2006; Schubert & Glänzel, 2007).
Table 2
H index of the authors
Author ID |
h index in WoS |
78120 |
8 |
72199 |
5 |
72149 |
5 |
78113 |
5 |
72174 |
3 |
72214 |
2 |
72186 |
3 |
72181 |
3 |
88128 |
5 |
78101 |
5 |
Own elaboration
Table 2 provides the indicators obtained by each author, taking data from the eleven years of study. The results show that the best-positioned author was 78120, who managed to obtain an h8 index, which means he has eight publications, each of which received at least eight citations. Next, five researchers obtained an h5 index, followed by authors who received at least three citations (h3) in three publications. Finally, the author with the lowest indicator was 72174, who only reached an index of two publications with at least two citations (h2).
4.8 Comparison of WoS and Google Scholar indicators
Some of the bibliometric indicators, such as citations and h index, however are limited since they only measure the Web of Science publications. We compared these results with the indicators obtained for the same authors in Google Scholar to broaden our view. Table 3 shows the results.
Table 3
Comparison between indicators, Faculty of Engineering
Author ID |
Total citations WoS |
Total citations Google |
h index in WoS |
h index in Google |
78120 |
124 |
618 |
8 |
14 |
72199 |
102 |
439 |
5 |
11 |
72149 |
64 |
0 |
5 |
0 |
78113 |
63 |
0 |
5 |
0 |
72174 |
65 |
357 |
2 |
6 |
72214 |
57 |
707 |
3 |
14 |
72186 |
52 |
434 |
3 |
12 |
72181 |
54 |
0 |
3 |
0 |
88128 |
44 |
0 |
5 |
0 |
78101 |
57 |
0 |
5 |
0 |
Own elaboration
There is a significant difference in the citation indicator and for the h index: first, citations for all the authors in Google Scholar considerably exceed those obtained in the WoS database. Author 72214 shows the most prominent difference; according to Google, he is among the best researchers, with 707 received citations, while he has only 57 in the WoS. The same happens with his h index: according to WoS, he has an h3, while according to Google, he reaches an h14, the same as the researchers who obtained the best indexes in the analysis of the Web of Science. Secondly, some authors do not have citations or h indicators because they do not have a profile created in the platform, so this information could not be recovered. Table 4 shows the comparison for the Faculty of Business.
Table 4
Comparison between indicators, Faculty of Business
Total citations WoS |
Total citations Google |
h index in WoS |
h index in Google |
|
72156 |
67 |
588 |
4 |
12 |
72272 |
60 |
0 |
2 |
0 |
72161 |
34 |
218 |
4 |
9 |
78129 |
16 |
76 |
2 |
5 |
72162 |
9 |
173 |
2 |
8 |
72233 |
9 |
0 |
1 |
0 |
Own elaboration
As it happened with the Faculty of Engineering, the rates of the Business School increased for all researchers; however, the best author of the WoS continues to rank in the first place according to Google Scholar data. On the other hand, author 72156 significantly increased the number of citations received, as well as the h index, becoming one of the best positioned.
4.9 Network Indicator
To complete the search for the most valuable researchers, we compared the network indicators to identify similarities among authors who possess the best indicators, as shown in table 5.
Table 5
Networks indicators in the Faculty of Engineering
Author ID |
Grade |
Co-authorship |
Network size |
Area |
78120 |
9 |
27 |
34 |
10 |
72199 |
10 |
35 |
46 |
3 |
72149 |
10 |
9 |
47 |
6 |
78113 |
8 |
10 |
33 |
9 |
72174 |
5 |
9 |
28 |
5 |
72214 |
6 |
12 |
15 |
6 |
72186 |
4 |
1 |
34 |
2 |
72181 |
4 |
22 |
29 |
8 |
88128 |
4 |
13 |
34 |
8 |
78101 |
2 |
0 |
15 |
5 |
Own elaboration
Table 5 shows that, in general, for the Faculty of Engineering, authors are related to six countries, on average they work in their research with 14 co-authors, their average network size is 31 researchers, while they relate to other six areas, different to the one of the principal author. Sixty percent of the researchers are related to five or more countries, and 100 percent have a network size of more than 15 researchers, while 10 percent make their publications alone and another 10 percent do it with just one co-author. Eighty percent work with at least five different areas of knowledge and 100 percent relate to other areas.
Table 5 shows that author 78120 has quite good indicators; for the degree, the indicator is equivalent to nine, which indicates that the author has relationships with various international researchers, in this case with nine countries. In addition, these researchers come from areas different from the author’s; as the area indicator shows, he has relations with ten disciplines different from his own. In the case of co-authors, the author has worked with 27 different researchers and has participated in conjunction with 34 researchers. This indicates that the author has an extensive network of work and information flow, making him one of the leading candidates to choose the name of the most valuable researcher.
Author 72199 has excellent network indicators, except for one, but that does not directly affect the researcher’s performance. In the first place, the author has relationships with 46 researchers in ten different countries, which indicates that their network of collaborators is quite broad; in the same way, the co-authorship indicator shows a high number of researchers who have collaborated with the author in his publications, 35 co-authors specifically. Finally, there is a low relation with other specialty areas since it is related to only three; this does not mean that it is a poor index; it only indicates that his field of knowledge is centered only in his area, and he does not address different issues. In the same way as the previous author, he is also a candidate for the name of the most valuable researcher.
Author 72149 has a network size of 47 researchers in ten different countries, implying that he has quite an extensive collaboration network; however, in his work, he has only shared signatures with nine researchers, which indicates that he has numerous participations such as co-author in other publications. Finally, this author is related to six areas different from the main one, so he is a candidate to be the most valuable researcher.
Table 6 contains each indicator of the authors, both of the Faculty of Engineering and Business of the University. Thus, it was possible to compare both groups and interpret characteristics that the authors share to establish factors that lead them to belong to the group of best researchers.
Table 6
Ranking of authors with the best indicators
Author ID |
Faculty |
Annual |
H Indicators |
Impact |
Network |
Co-authorship |
Network |
Investigation |
78120 |
Engineering |
4.2 |
8 |
1 |
9 |
27 |
34 |
10 |
72199 |
Engineering |
2.7 |
5 |
2 |
10 |
35 |
46 |
3 |
72149 |
Engineering |
2.3 |
5 |
2 |
10 |
9 |
47 |
6 |
72156 |
Business |
1.3 |
2 |
2 |
4 |
4 |
19 |
6 |
72272 |
Business |
1 |
4 |
3 |
3 |
1 |
3 |
6 |
Own elaboration
From table 6, it was possible to establish the necessary characteristics so that a researcher could develop optimally in teaching research. Hence, table 6 shows that 80 percent of researchers have high impact factors, between one and two, which means that the best researchers share the importance of publishing in high-impact journals. Faculty of Engineering researchers have a much higher network size index than Faculty of Business researchers, perhaps a critical point in the observed productivity. Likewise, the number of countries with which the authors are associated is high for the Faculty of Engineering and lower for the Faculty of Business.
4.10 Multiple linear regression analysis
We elaborated a correlation matrix before performing the regression analysis. We correlated the collaborative network variable with the grade indicator, co-authorship, external network size, research area (independent variable), and annual productivity (dependent variable). We used IBM SPSS software to obtain the correlation matrix, as shown in table 7.
Table 7
Correlation matrix
Annual productivity |
Network grade indicator |
Co-authorship indicator |
External |
Investigation area |
|
Annual productivity |
1 |
||||
Grade Indicator |
0,806** |
1 |
|||
Co-authorship indicator |
0,700** |
0,663** |
1 |
||
External network size |
0,642** |
0,837** |
0,537** |
1 |
|
Investigation area |
0,422** |
0,263** |
0,136 |
0,106 |
1 |
** The correlation is significant at level 0,01 (bilateral)
Own elaboration
Table 7 indicates that the variables that have a significant correlation with the researchers’ annual scientific productivity are the grade indicator (0,8), co-authorship (0,7), network size (0,642), and research area (0,422). Given these correlations, all variables affect the productivity of researchers, which explains that researchers with higher network indicators have higher annual productivity.
Finally, the correlation between the research area and the degree of the network was found to be (0.26**), indicating that an increase or decrease in the research areas will influence the increase or decrease of the network of countries with which the authors are related.
After analyzing the correlation matrix, we performed the multiple regression analysis. Results are shown in table 9.
Table 8
Model summary
Model |
R |
R square |
R squared adjusted |
1 |
0,867* |
0,752 |
0,747 |
* Predictions: (Constant), research area, external network size, co-authorship
indicator, network-grade indicator.
Own elaboration
Table 8 shows the summary of the model, which shows that the model explains 74,7 percent of the variability observed in the annual productivity of researchers analyzed. The variables that influence the variability of the annual productivity of researchers that show a significant effect are the network grade indicator, with a high level of significance (0,000), the co-authorship indicator, with a high level of significance (0,000), and the research area, with a high level of significance (0,000) with the annual productivity of researchers (see table 9).
Finally, the size of the external network does not influence productivity; that is, increasing relationships outside the research group will not help improve the researcher’s productivity.
Table 9
Multiple linear regression
Indicators |
Non-standardized coefficients |
Standardized coefficients |
||||
B |
Dev. Error |
Beta |
T |
Sig. |
||
Model 1(Constant) |
-0,31 |
0,106 |
-0,296 |
0,767 |
||
Network grade indicator |
0,177 |
0,25 |
0,538 |
7,044 |
0,000 |
|
Co-authorship indicator |
0,27 |
0,004 |
0,312 |
6,608 |
0,000 |
|
External network size |
-0,00006022 |
0,005 |
-0,001 |
-0,13 |
0,99 |
|
Research area |
0,087 |
0,014 |
0,239 |
6,38 |
0,000 |
Own elaboration
Considering the analysis of correlations and regression where the variables that influence productivity were obtained (network grade indicator, co-authorship and research) we can conclude that the authors selected in the ranking of best researchers have pretty good network variables, verifying that collaboration networks have a considerable influence on the productivity of the most valuable researchers.
5. Discussion
Our research allowed us to establish a methodology to determine each step necessary to obtain the results of the indicators and thus establish the standard for the most valuable researcher.
This investigation proved what Lotka said: that most of the researches were done by a small group of researchers, finding that many researchers only have one article published in the WoS database, this is how 57 percent of the authors of the Faculty of Engineering only have one article published while 58 percent of the Faculty of Business have one.
The analysis of results with descriptive statistics highlights that there are notorious differences between both faculties, starting with the number of researchers (70 percent corresponding to authors of the Faculty of Engineering and only 30 percent to the Faculty of Business). The same happens with the number of publications: between 2008 and 2018, 74,38 percent correspond to Engineering and 25,62 percent to Business. Therefore, based on Delgado & Cabezas (2012), working with both faculties separately was necessary because the research areas are not comparable.
Comparing our results with those of other studies shows that those researchers with high productivity indicators manage to position themselves as high-level producers. The same happens with the h index of the authors since, for the most part, having a high h index is evidence that the researcher has good production indicators.
The comparison between the citation indicators and an h index of Google Scholar and WoS shows similarities in terms of the trend of researchers. Those who have indicators above the rest in WoS have even better ones in in Google Scholar, confirming what was said by Orduña-Malea et al. (2015) and Harzing & Alakangas (2016). This is because Google Scholar searches for all the articles by the authors, indexed or not, to generate the indicators shown for them, so there is no adequate quality filter for the publications. In this way, it became necessary to search the researchers according to the criteria of the Web of Science and then compare the results with Google Scholar.
From the regression analysis, it is possible to demonstrate that network indicators influence researchers’ annual productivity. Compared with other research, such as the one carried out by García (2013), similar results are obtained regarding the correlations between collaborative network variables. For this case, we have that degree indicator (0,8), co-authorship (0,7), network size (0,642), and research area (0,422) have strong correlations with productivity. However, after obtaining the multiple regression analysis, only three of the four variables are significant to explain the variability of productivity, leaving outside the size of the external network. This may be due to correlating with another variable, so it is discarded only for the variable productivity, which is why it is advisable to perform a more extensive analysis to validate the information obtained.
6. Conclusions and Recommendations
Some precedents allow knowing the criteria necessary to achieve academic production efficiently. The main indicators that allow us to disclose the standard with which the most valuable researchers work are the productivity indicator, h index, and impact factor. Meanwhile, for collaborative networks, network size, research area, and co-authorship are the indicators that help explain the success of these researchers, or like it has been called, the most valuable researcher (MVR).
In the Faculty of Engineering case, the h index shows its best results in three academics, with an index of between five and eight; in turn, according to a productivity of at least 2,3 annual publications. Another factor in which the authors who have these indicators agree is to publish their research in journals that have a high impact factor; these are listed as Q1 and Q2 journals.
In the Faculty of Business case, two authors were selected as the most valuable, having an h index between two and four, while their productivity is only one annual publication. They publish in journals with the average impact factor cataloged as Q2 and Q3.
The comparison made between the WoS and Google Scholar indicators results in an evident improvement of the indicators given by Google, however, the ranking of authors is not strongly affected, because the authors stayed in their ranking positions.
The analysis of collaborative networks shows that authors selected as the most valuable have a wide area of work, and the vast majority participate in at least six areas within their area of knowledge or related to it. This is reflected in the size of the network they manage since the authors of the Faculty of Engineering have relations with at least 34 researchers in at least nine countries. Authors of Businesses handle fewer interrelations varying between 3 and 19 researchers.
The analysis of correlations and linear regression verify the hypothesis obtained through the descriptive analysis of network indicators. Because the variable collaborative network has a direct relationship with the productivity of researchers, it is a fact that indicators of a network’s degree, co-authorship and research area are variables that affect productivity. In contrast, the variable “external network size” was discarded by the regression test.
Another recommendation is to publish as the first option in journals whose impact factor is Q1 or Q2 since the most valuable researchers share the characteristic of making their publications in high-impact journals.
This study can be replicated in its entirety for any organization that needs to know its MVRs; however, we recommend extending the study to different databases because currently there are new platforms that gather research; Scopus is relevant for Business and Administration, for example. Over the years many researchers have migrated to this platform, so adding it would be an advance in the search for the best researchers and explaining how to get to be an MVR.
As proposals for future research on academic research, we propose the option of including study groups of the National Committees of Science and Technology, such as the National Commission for Scientific and Technological Research in Chile, determining whether it is influential or not.
Based on the preceding, it would be timely and beneficial for future research: (a) To analyze in a more exhaustive way the variables and their correlations, determining if there are correlations with variables not considered in the linear regression analysis; (b) Analyze how to include the effect of young researchers versus senior researchers, considering the productive life of researchers; (c) Analyze how the databases enumerate the authors or co-authors, according to their name, specialty areas, author of correspondence, and verify if there are differences with reality; (d) Identify the effect of “initiator” or “inciter”; regarding researchers with high citation rates, but with low productivity also establish how they influence the publication as author or co-author; and (e) Expand the study to compare public/private institutions throughout a region or country, by specialties.
This study of the most valuable researcher presents an efficient way to establish scientific productivity within an Academic Unit or a University and how to contribute to improving the quality of research through sponsorship and support of collaborative networks.
Bordons, M. (1999). Evaluación de la actividad científica a través de indicadores bibliométricos. Revista Española de Cardiología, 52, 790-800.
Braun, T., Glänzel, W., & Schubert, A. (2005). A Hirsch-type index for journals. The Scientist, 19, 8.
Cargill, M., & Burgess, S. (Eds.) (2017). Publishing research in English as an additional language: Practices, pathways and potentials. University of Adelaide Press.
Chaparro, N., & Rojas-Galeano, S. (2021) Revealing the Research Landscape of Master’s Degrees via Bibliometric Analyses. Library Philosophy and Practice (e-journal), 5351. https://digitalcommons.unl.edu/libphilprac/5351
Cole, F. J., & Eales, N. B. (1917). The history of comparative anatomy: Part I.—A statistical analysis of the literature. Science Progress, 11, 578-596.
Delgado, E., & Cabezas-Clavijo, Á. (2012). Google Scholar Metrics: an Unreliable Tool for Assessing Scientific Journals. El Profesional de la Información, 21, 419-427.
Franco-López, A. F., Sanz-Valero, J., & Fernández, J. M. C. (2016). Publicar en castellano, o en cualquier otro idioma que no sea inglés, negativo para el factor de impacto y citaciones. Journal of Negative and No Positive Results, 1, 65-70.
García, A. (2013). Las redes de colaboración científica y su efecto en la productividad. Un análisis bibliométrico. Investigación Bibliotecológica, 27, 159-175.
Gingras, Y. (1996). Bibliometric analysis of funded research. A feasibility study. Centre interuniversitaire de recherche sur la science et la technologie (CIRST), Canadá.
Gross, P. L., & Gross, E. M. (1927). College libraries and chemical education. Science, 66, 385-389.
Harzing, A. W., & Alakangas, S. (2016). Google Scholar, Scopus and the Web of Science: a Longitudinal and Cross-Disciplinary Comparison. Scientometrics, 106, 787-804. https://doi.org/10.1007/s11192-015-1798-9
Hulme, E. W. (1923). Statistical bibliography in relation to the growth of modern civilization. Grafton.
Jacsó, P. (2008). The Pros and Cons of Computing the h-Index Using Web of Science. Online Information Review, 32, 673-688.
King, J. (1987). A Review of Bibliometric and Other Science Indicators and their Role in Research Evaluation. Journal of Information Science, 13, 261-276.
Mitra, P. (2006). Hirsch-Type Indices for Ranking Institutions Scientific Research Output. Current Science, 91, 1439.
Okubo, Y. (1997), Indicateurs bibliométriques et analyse des systèmes de recherché, Organisation for Economic Co-operation and Development (OECD).
Orduña-Malea, E., Ayllón, J. M., Martín Martín, A., & Delgado López-Cózar, E. (2015). Aplicaciones métricas de Google Scholar para la evaluación del impacto científico. En IV Jornadas de Intercambio y Reflexión acerca de la Investigación en Bibliotecología, Universidad Nacional de La Plata, Argentina, 1-12.
Patra, S. K., Bhattacharya, P., & Verma, N. (2006). Bibliometric Study of Literature on Bibliometrics. DESIDOC Journal of Library & Information Technology, 26. https://doi.org/10.14429/djlit.26.1.3672
Pritchard, A. (1969). Statistical bibliography or bibliometrics. Journal of Documentation, 25, 348–349.
Rinia, E. J., Van Leeuwen, T. N., Van Vuren, H. G., & Van Raan, A. F. (1998). Comparative Analysis of a Set of Bibliometric Indicators and Central Peer Review Criteria: Evaluation of Condensed Matter Physics in the Netherlands. Research Policy, 27, 95–107. https://doi.org/10.1016/S0048-7333(98)00026-2
Roig-Tierno, N., Gonzalez-Cruz, T. F., & Llopis-Martinez, J. (2017). An Overview of Qualitative Comparative Analysis: A Bibliometric Analysis. Journal of Innovation & Knowledge, 2, 15–23.
Roldan, C. S., Chaparro, N., & Rojas-Galeano, S. (2019). Análisis bibliométrico de la Revista Ingeniería (2010-2017). Ingeniería, 24(2), 96-115. https://revistas.udistrital.edu.co/index.php/reving/article/view/14678
Schauss, M. A. (1998). Disease indicator analysis system U.S. Patent No. 5,746,204, U.S. Patent and Trademark Office.
Schubert, A., & Glänzel, W. (2007). A Systematic Analysis of Hirsch-Type Indices for Journals. Journal of Informetrics, 1, 179–184. https://doi.org/10.1016/j.joi.2006.12.002
Sengupta, I. N. (1992). Bibliometrics, Informetrics, Scientometrics and Librametrics: an Overview. Libri, 42, 75–98.
White, H. D., & McCain, K. W. (1998). Visualizing a Discipline: An Author Co‐Citation Analysis of Information Science, 1972–1995. Journal of the American Society for Information Science, 49, 327–355.
Zupic, I., & Čater, T. (٢٠١٥). Bibliometric Methods in Management and Organization. Organizational Research Methods, 18, 429–472. https://doi.org/10.1177/1094428114562629
* Todos autores han contribuido con la misma intensidad en el diseño, obtención de datos, analisis, revisión crítica de su contenido y aprobación final de la versión publicada.
**Correos electrónicos: camilo.pena@ucentral.cl; leonardoconchar@gmail.com; eforcael@ubiobio.cl;
gegarces@ubiobio.cl