Universities’ Research Performance: Evaluation Issues
Abstract
The need to review the universities’ research performance evaluation, which is based on scientometric indicators is raised in the paper. The attention is drawn to an objective understanding of university rankings, which in many countries are the basis for financial support of higher education institutions without taking into account their successful implementation of regional and national social needs. The Ukrainian regulative requirements concerning scholarly publishing in the Scopus and Web of Science citation databases of the faculty, higher education institutions research performance, research journals functioning are given. The dynamics and presentation of the Ukrainian classical and pedagogical universities scholarly publishing in the research area of Education and Educational Research during 2016–2020 covered by the Web of Science Core Collection database is analysed. The specifics of the Social Sciences and Humanities, which should be considered during the development of effective methods and innovative approaches of universities research performance evaluation, is enlightened, in particular, the importance of monographs and books; pronounced national and regional significance; citing culture giving the medium and long term perspective of the Social Sciences and Humanities conceptual bases; multilingualism of publications; the importance of translating publications to achieve open access; interdisciplinary nature of research; predominance of single authorship; ensuring objective research performance evaluation within one field, creation of a polycentric multilingual research evaluation system.
Downloads
References
Cabinet of Ministers of Ukraine. (1994, August 23). Pro stypendii Prezydenta Ukrainy dlia molodykh uchenykh [On Scholarships of the President of Ukraine for Young Scientists] (582). Retrieved from https://zakon.rada.gov.ua/laws/show/582-94-п [in Ukrainian]
Cabinet of Ministers of Ukraine. (1999, August 13). Pro zatverdzhennia Polozhennia pro atestatsiiu naukovykh pratsivnykiv [On Approval of the Regulation on Certification of Researchers] (1475). Retrieved from https://zakon.rada.gov.ua/laws/show/1475-99-п [in Ukrainian]
Cabinet of Ministers of Ukraine. (2016, March 23). Pro zatverdzhennia Poriadku pidhotovky zdobuvachiv vyshchoi osvity stupenia doktora filosofii ta doktora nauk u zakladakh vyshchoi osvity (naukovykh ustanovakh) [On Approval of the Procedure for PhD and DSc Students Preparation in Higher Education Institutions (Research Institutions)] (261). Retrieved from https://zakon.rada.gov.ua/laws/show/261-2016-п [in Ukrainian]
Cabinet of Ministers of Ukraine. (2018a, November 21). Pro derzhavni imenni stypendii naikrashchym molodym vchenym dlia uvichnennia podii Revoliutsii Hidnosti ta vshanuvannia podvyhu Heroiv Ukrainy – Heroiv Nebesnoi Sotni [On State Nominal Scholarships for the Best Young Scientists to Perpetuate the Events of the Revolution of Dignity and Honour the Heroism of the Heroes of Ukraine – Heroes of the Heavenly Hundred] (1017). Retrieved from https://zakon.rada.gov.ua/laws/show/1017-2018-п [in Ukrainian]
Cabinet of Ministers of Ukraine. (2018b, August 22). Deiaki pytannia provedennia derzhavnoi atestatsii zakladiv vyshchoi osvity v chastyni provadzhennia nymy naukovoi (naukovo-tekhnichnoi) diialnosti [Some Issues of State Certification of Higher Education Institutions in Terms of Their Scientific (Scientific and Technical) Activities] (652). Retrieved from https://zakon.rada.gov.ua/laws/show/652-2018-п [in Ukrainian]
Cabinet of Ministers of Ukraine. (2019a, December 27). Pro zatverdzhennia Poriadku konkursnoho vidboru ta finansuvannia Natsionalnym fondom doslidzhen proektiv z vykonannia naukovykh doslidzhen i rozrobok [On Approval of the Procedure for Competitive Selection and Financing of R&D Projects by the National Research Fund] (1170). Retrieved from https://zakon.rada.gov.ua/laws/show/1170-2019-п [in Ukrainian]
Cabinet of Ministers of Ukraine. (2019b, March 6). Pro provedennia eksperymentu z prysudzhennia stupenia doktora filosofii [On Conducting an Experiment to Award the PhD Degree] (167). Retrieved from https://zakon.rada.gov.ua/laws/show/167-2019-п [in Ukrainian]
Cabinet of Ministers of Ukraine. (2021, March 24). Pro vnesennia zmin do postanovy Kabinetu Ministriv Ukrainy vid 30 hrudnia 2015 r. № 1187 [On Amendments to the Decree of the Cabinet of Ministers of Ukraine of December 30, 2015 No 1187] (365). Retrieved from https://zakon.rada.gov.ua/laws/show/365-2021-п [in Ukrainian]
Kremen, V. H., Luhovyi, V. I., Reheilo, I. Yu., Bazeliuk, N. V., & Bazeliuk, O. V. (2020). Openness, Digitalization and Evaluation in Research: General and Special Issues for Social Studies and Humanities. Information Technologies and Learning Tools, 80(6), 243–266. Retrieved from https://doi.org/10.33407/itlt.v80i6.4155 [in Ukrainian]
Luhovyi, V. I., Reheilo, I. Yu., Bazeliuk, N. V., & Bazeliuk, O. V. (2019). Global Digitisation of the Education and Research Area and Challenges in Modernising the Scientific Periodicals of NAES of Ukraine. Information Technologies and Learning Tools, 73(5), 264–283. Retrieved from https://doi.org/10.33407/itlt.v73i5.3366 [in Ukrainian]
Ministry of Education and Science of Ukraine. (2015, December 14). Pro zatverdzhennia Polozhennia pro provedennia Ministerstvom osvity i nauky Ukrainy Konkursu proektiv naukovykh robit ta naukovo-tekhnichnykh (eksperymentalnykh) rozrobok molodykh vchenykh, yaki pratsiuiut (navchaiutsia) u vyshchykh navchalnykh zakladakh ta naukovykh ustanovakh, shcho nalezhat do sfery upravlinnia Ministerstva, podalshe vykonannia tsykh robit i rozrobok za rakhunok koshtiv zahalnoho fondu derzhavnoho biudzhetu ta upravlinnia nymy [On Approval of the Regulation on Carrying out by the Ministry of Education and Science of Ukraine of the Competition of R&D Projects of Young Scientists, Young Scientists Who Work (Study) in Higher Education and Research Institutions, Which are Subordinated to the Ministry, Further Implementation of Those R&D at the Expense of the State Budget and Managing Them] (1287). Retrieved from https://zakon.rada.gov.ua/laws/show/z0015-16 [in Ukrainian]
Ministry of Education and Science of Ukraine. (2016, January 14). Pro zatverdzhennia Poriadku prysvoiennia vchenykh zvan naukovym i naukovo-pedahohichnym pratsivnykam [On Approval of the Procedure of Assigning Academic Titles to Researchers and the Faculty] (13). Retrieved from https://zakon.rada.gov.ua/laws/show/z0183-16 [in Ukrainian]
Ministry of Education and Science of Ukraine. (2018a, January 15). Pro zatverdzhennia Poriadku formuvannia Pereliku naukovykh fakhovykh vydan Ukrainy [On Approval of the Procedure of Forming the List of Scientific Professional Periodicals of Ukraine] (32). Retrieved from https://zakon.rada.gov.ua/laws/show/z0148-18 [in Ukrainian]
Ministry of Education and Science of Ukraine. (2018b, May 3). Pro zatverdzhennia Kryteriiv konkursnoho vidboru vykonavtsiv derzhavnoho zamovlennia na pidhotovku mahistriv u zakladakh vyshchoi osvity, shcho nalezhat do sfery upravlinnia MON [On Approval of the Criteria of Competitive Selection of the State Order Executors for Master Degree Students Preparation in Higher Education Institutions Subordinated to the Ministry of Education and Science of Ukraine] (445). Retrieved from https://zakon.rada.gov.ua/laws/show/z0625-18 [in Ukrainian]
Ministry of Education and Science of Ukraine. (2019a, March 12). Pro derzhavnu atestatsiiu zakladiv vyshchoi osvity v chastyni provadzhennia nymy naukovoi (naukovo-tekhnichnoi) diialnosti [On the State Certification of Higher Education Institutions in Terms of Their Scientific (Scientific and Technical) Activities] (338). Retrieved from https://zakon.rada.gov.ua/laws/show/z0688-19 [in Ukrainian]
Ministry of Education and Science of Ukraine. (2019b, April 19). Pro zatverdzhennia Polozhennia pro Komisiiu Ministerstva osvity i nauky Ukrainy z vidboru pretendentiv na zdobuttia Premii Verkhovnoi Rady Ukrainy molodym uchenym, imennykh stypendii Verkhovnoi Rady Ukrainy dlia molodykh uchenykh – doktoriv nauk ta rozghliadu naukovykh zvitiv stypendiativ [On Approval of the Regulation on the Commission of the Ministry of Education and Science of Ukraine on Selection of Candidates for the Verkhovna Rada of Ukraine Award for Young Scientists, Nominal Scholarships of the Verkhovna Rada of Ukraine for Young Scientists with DSc Degree and Reviewing Scientific Reports of Scholars] (522). Retrieved from https://zakon.rada.gov.ua/laws/show/z0512-19 [in Ukrainian]
Ministry of Education and Science of Ukraine. (2019c, September 23). Pro opublikuvannia rezultativ dysertatsii na zdobuttia naukovykh stupeniv doktora i kandydata nauk [On Publishing the Results of the Theses for Candidate and Doctor of Sciences Degree] (1220). Retrieved from https://zakon.rada.gov.ua/laws/show/z1086-19 [in Ukrainian]
Ministry of Education and Science of Ukraine. (2020, November 18). Pro vnesennia zmin do Poriadku formuvannia Pereliku naukovykh fakhovykh vydan, zatverdzhenoho nakazom Ministerstva osvity i nauky Ukrainy vid 15 sichnia 2018 roku № 32 [On Amendments to the Procedure of Forming the List of Scientific Professional Periodicals of Ukraine, Approved by the Ministry of Education and Science of Ukraine Order on January 15, 2018 No 32] (1437). Retrieved from https://zakon.rada.gov.ua/laws/show/z0141-21 [in Ukrainian]
Ministry of Education and Science of Ukraine. (2021, March 17). Pro vnesennia zmin do Poriadku prysvoiennia vchenykh zvan naukovym i naukovo-pedahohichnym pratsivnykam [On Amendments to the Procedure of Assigning Academic titles to Researchers and the Faculty] (322). Retrieved from https://zakon.rada.gov.ua/laws/show/z0646-21 [in Ukrainian]
Center for International Projects «Euroeducation». (2020, June 7). Nezalezhne otsiniuvannia universytetiv: akademichnyi reitynh «Top-200 Ukraina 2020» [Independent Evaluation of Universities: Academic Ranking «Top-200 Ukraine 2020”]. Retrieved from http://www.euroosvita.net/index.php/?category=1&id=6556 [in Ukrainian]
Abstracts for the NordForsk Workshop on Bibliometrics for the Social Sciences and Humanities. (2012). Retrieved from https://blogs.helsinki.fi/nordforskssh2012/files/2012/03/NordForsk-SSH-Workshop-abstracts1.pdf [in English]
Australian Research Council. (2019, October 10). Excellence in Research for Australia. Retrieved from https://www.arc.gov.au/excellence-research-australia [in English]
Belcher, B., Rasmussen, K., Kemshaw, M., & Zornes, D. (2015). Defining and assessing research quality in a transdisciplinary context. Research Evaluation, 25 (1), 1–17. Retrieved from https://doi.org/10.1093/reseval/rvv025 [in English]
Bunia, R. (2016). Quotation statistics and culture in literature and in other humanist disciplines. In M. Ochsner, S.E. Hug, H.-D. Daniel (Eds), Research Assessment in the Humanities. Towards Criteria and Procedures (pp. 133–148). Springer International Publishing: Cham, Switzerland. Retrieved from https://doi.org/10.1007/978-3-319-29016-4_11 [in English]
Claeys-Kulik, A.-L. (2021, June 10). EU university policy needs to support, not direct. Research Professional News. Retrieved from https://www.researchprofessionalnews.com/rr-news-europe-views-of-europe-2021-6-eu-university-policy-needs-dialogue-not-diktat/ [in English]
Clarivate. (2021). Web of Science Core Collection. Retrieved from https://www.webofscience.com/wos/woscc/basic-search [in English]
Delgado-López-Cózar, E., Ràfols, I., & Abadal, E. (2021). Letter: A call for a radical change in research evaluation in Spain. Profesional De La Información, 30(3). Retrieved from https://doi.org/10.3145/epi.2021.may.09 [in English]
García-Zorita, C., Marugán, S., de Filippo, D., & Sanz-Casado, E. (2021). Highly Cited Papers at the Spanish Domestic Level. Frontiers in Research Metrics and Analytics, 6. Retrieved from https://doi.org/10.3389/frma.2021.651991 [in English]
Gómez-Ferri, J., González-Alcaide, G., & LLopis-Goig, R. (2019). Measuring dissatisfaction with coauthorship: An empirical approach based on the researchers’ perception. Journal of Informetrics, 13 (4), 100980. Retrieved from https://doi.org/10.1016/j.joi.2019.100980 [in English]
Hicks, D. (2004). The Four Literatures of Social Science. In H.F. Moed, W. Glänzel, U. Schmoch (Eds), Handbook of Quantitative Science and Technology Research (pp. 473–496). Springer, Dordrecht. Retrieved from https://doi.org/10.1007/1-4020-2755-9_22 [in English]
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520 (7548), 429–431. Retrieved from https://doi.org/10.1038/520429a [in English]
Hug, S., Ochsner, M., & Daniel, H. (2013). Criteria for assessing research quality in the humanities: a Delphi study among scholars of English literature, German literature and art history. Research Evaluation, 22 (5), 369–383. Retrieved from https://doi.org/10.1093/reseval/rvt008 [in English]
Joseph Jeyaraj, J., Wald, N. & Harland, T. (2021). Higher education teachers’ experiences of becoming research active: striving for university status in the Global South. Asia Pacific Education Review. Retrieved from https://doi.org/10.1007/s12564-021-09688-8 [in English]
Korytkowski, P., & Kulczycki, E. (2019). Publication counting methods for a national research evaluation exercise. Journal of Informetrics, 13 (3), 804–816. Retrieved from https://doi.org/10.1016/j.joi.2019.07.001 [in English]
Kulczycki, E., Engels, T., Pölönen, J., Bruun, K., Dušková, M., & Guns, R. et al. (2018). Publication patterns in the social sciences and humanities: evidence from eight European countries. Scientometrics, 116 (1), 463–486. Retrieved from https://doi.org/10.1007/s11192-018-2711-0 [in English]
Larivière, V., Kiermer, V., MacCallum, C.J., McNutt, M., Patterson, M., Pulverer, B., Swaminathan, S., Taylor, S., & Curry, S. (2016). A simple proposal for the publication of journal citation distributions. bioRxiv. Retrieved from https://doi.org/10.1101/062109 [in English]
Nederhof, A. (2006). Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review. Scientometrics, 66 (1), 81–100. Retrieved from https://doi.org/10.1007/s11192-006-0007-2 [in English]
Ochsner, M., Hug, S. E., & Daniel, H.-D. (Eds). (2016). Research Assessment in the Humanities. Towards Criteria and Procedures. Springer, Cham. Retrieved from https://doi.org/10.1007/978-3-319-29016-4 [in English]
Piron, F., Olyhoek, T., Vilchis, I. L., Smith, I., & Liré, Z. (2021a). Saying ‘No’ to Rankings and Metrics. In B. Hall, R. Tandon (Eds), Socially Responsible Higher Education. International Perspectives on Knowledge Democracy (pp. 80–91). Leiden: Brill. Retrieved from https://doi.org/10.1163/9789004459076_007 [in English]
Piron, F., Olyhoek, T., Vilchis, I. L., Smith, I., & Liré, Z. (2021b). Trying to Say ‘No’ to Rankings and Metrics. In B. Hall, R. Tandon (Eds), Socially Responsible Higher Education. International Perspectives on Knowledge Democracy (pp. 276–287). Leiden: Brill. Retrieved from https://doi.org/10.1163/9789004459076_023 [in English]
Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010, 26 October). Altmetrics: A manifesto. Retrieved from http://altmetrics.org/manifesto [in English]
Research Excellence Framework. (2014). Retrieved from https://www.ref.ac.uk/2014/ [in English]
San Francisco Declaration on Research Assessment. (2012, December 16). Retrieved from https://sfdora.org/read/ [in English]
Sivertsen, G. (2018). The Norwegian Model in Norway. Journal of Data and Information Science, 3 (4), 3–19. Retrieved from https://doi.org/10.2478/jdis-2018-0017 [in English]
Sivertsen, G. (2019). Understanding and Evaluating Research and Scholarly Publishing in the Social Sciences and Humanities (SSH). Data and Information Management, 3 (2), 61–71. Retrieved from https://doi.org/10.2478/dim-2019-0008 [in English]
Szomszor, M., Adams, J., Fry, R., Gebert, C., Pendlebury, D., Potter, R., & Rogers, G. (2021). Interpreting Bibliometric Data. Frontiers in Research Metrics and Analytics, 5. Retrieved from https://doi.org/10.3389/frma.2020.628703 [in English]
Terheggen, Ph. (2021, April 15). Re-evaluating reward and recognition in science. Germany, the Netherlands and other leading research nations have the right idea in moving away from citations as the defining metric. Elsevier Connect. Retrieved from https://www.elsevier.com/connect/re-evaluating-reward-and-recognition-in-science [in English]
The PIE News. (2021, June 4). Michael Murphy, President, European University Association. Retrieved from https://thepienews.com/pie-chat/michael-murphy-europe/ [in English]
University Wankings. (2021). Why Are Our Rankings So White? In B. Hall, R. Tandon (Eds), Socially Responsible Higher Education. International Perspectives on Knowledge Democracy (pp. 67–79). Leiden: Brill. Retrieved from https://doi.org/10.1163/9789004459076_006 [in English]
Van den Akker, W. (2016). Yes We Should; Research Assessment in the Humanities. In M. Ochsner, S.E. Hug, H.-D. Daniel (Eds), Research Assessment in the Humanities. Towards Criteria and Procedures (pp. 23–29). Springer International Publishing: Cham, Switzerland. Retrieved from https://doi.org/10.1007/978-3-319-29016-4_3 [in English]
Van Leeuwen, T. (2013). Bibliometric research evaluations, Web of Science and the Social Sciences and Humanities: a problematic relationship? Bibliometrie – Praxis und Forschung, (2). Retrieved from https://doi.org/10.5283/bpf.173 [in English]
Copyright (c) 2021 Pedagogical Discourse
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
