The meritocratic degree of the Academic evaluation in Italy Should artificial intelligence be used in this context: Difference between revisions
(Created page with "{{DISPLAYTITLE:The meritocratic degree of the Academic evaluation in Italy. Should artificial intelligence be used in this context?}}{{#seo: |title=The meritocratic degree of the Academic evaluation in Italy. Should artificial intelligence be used in this context? - Top Italian Scientists Journal |description=The increasing emphasis on publication quantity over quality in academic evaluation has raised concerns about the integrity of meritocratic assessments in Italy. |k...") |
No edit summary |
||
| Line 45: | Line 45: | ||
''Correspondence to'': Salvatore Chirumbolo; Department of Engineering for Innovation Medicine; University of Verona, Italy; Strada Le Grazie 8, 37134 Verona; Tel. +390458027645 ; e-mail salvatore.chirumbolo@univr.it | ''Correspondence to'': Salvatore Chirumbolo; Department of Engineering for Innovation Medicine; University of Verona, Italy; Strada Le Grazie 8, 37134 Verona; Tel. +390458027645 ; e-mail salvatore.chirumbolo@univr.it | ||
==Abstract== | ==Abstract== | ||
The increasing emphasis on publication quantity over quality in academic evaluation has raised concerns about the integrity of meritocratic assessments in Italy. This study explores the impact of Letters to the Editor (LTEs) and Correspondences on reputation indices, such as the h-index, and proposes an algorithm to minimize their disproportionate influence. The proliferation of AI tools, including machine learning and chatbots, has further complicated academic evaluation by enabling rapid manuscript production, potentially inflating publication counts without corresponding research depth. This paper introduces a weighted algorithm that adjusts the h-index by assigning lower weights to LTEs and Correspondences, based on predefined parameters. Using example datasets, the adjusted h-index significantly reduced scores compared to the original, reflecting more accurate merit assessments. Statistical analysis revealed a high correlation between original and adjusted h-indices (Pearson r = 0.98, Spearman r = 0.94), yet highlighted a weak correlation between letter contributions and rank order (r = -0.04). Paired t-tests confirmed significant differences between the two indices (p = 0.0393). The proposed method effectively penalizes superficial publications while preserving the value of full-length research articles. This approach offers a rigorous and mathematically grounded framework for evaluating academic productivity, particularly in high-stakes settings like Italy’s National Scientific Qualification competitions. Furthermore, integrating AI tools into evaluation systems could enhance transparency and fairness, reducing biases and legal disputes related to subjective judgments. Future research should refine these metrics for broader disciplinary applications, ensuring a balanced assessment of academic contributions in the era of artificial intelligence and automated publishing tools. | The increasing emphasis on publication quantity over quality in academic evaluation has raised concerns about the integrity of meritocratic assessments in Italy. This study explores the impact of Letters to the Editor (LTEs) and Correspondences on reputation indices, such as the h-index, and proposes an algorithm to minimize their disproportionate influence. The proliferation of AI tools, including machine learning and chatbots, has further complicated academic evaluation by enabling rapid manuscript production, potentially inflating publication counts without corresponding research depth. This paper introduces a weighted algorithm that adjusts the h-index by assigning lower weights to LTEs and Correspondences, based on predefined parameters. Using example datasets, the adjusted h-index significantly reduced scores compared to the original, reflecting more accurate merit assessments. Statistical analysis revealed a high correlation between original and adjusted h-indices (Pearson r = 0.98, Spearman r = 0.94), yet highlighted a weak correlation between letter contributions and rank order (r = -0.04). Paired t-tests confirmed significant differences between the two indices (p = 0.0393). The proposed method effectively penalizes superficial publications while preserving the value of full-length research articles. This approach offers a rigorous and mathematically grounded framework for evaluating academic productivity, particularly in high-stakes settings like Italy’s National Scientific Qualification competitions. Furthermore, integrating AI tools into evaluation systems could enhance transparency and fairness, reducing biases and legal disputes related to subjective judgments. Future research should refine these metrics for broader disciplinary applications, ensuring a balanced assessment of academic contributions in the era of artificial intelligence and automated publishing tools. | ||
==Declarations== | ==Declarations== | ||
| Line 57: | Line 55: | ||
===Author Contributions=== | ===Author Contributions=== | ||
Conceptualization, Data curation, Software, Manuscript writing, reviewing and submitting. | Conceptualization, Data curation, Software, Manuscript writing, reviewing and submitting. | ||
==References== | ==References== | ||
Revision as of 16:47, 3 January 2025
| Published |
| January 4, 2025 |
| Title |
| The meritocratic degree of the Academic evaluation in Italy. Should artificial intelligence be used in this context? |
| Authors |
| Salvatore Chirumbolo. |
| DOI |
| 10.62684/ANCA8483 |
| Keywords |
| Artificial intelligence, academic evaluation in Italy. |
| Downloads |
 |
Department of Engineering for Innovation Medicine, University of Verona, Verona, Italy
Correspondence to: Salvatore Chirumbolo; Department of Engineering for Innovation Medicine; University of Verona, Italy; Strada Le Grazie 8, 37134 Verona; Tel. +390458027645 ; e-mail salvatore.chirumbolo@univr.it
Abstract
The increasing emphasis on publication quantity over quality in academic evaluation has raised concerns about the integrity of meritocratic assessments in Italy. This study explores the impact of Letters to the Editor (LTEs) and Correspondences on reputation indices, such as the h-index, and proposes an algorithm to minimize their disproportionate influence. The proliferation of AI tools, including machine learning and chatbots, has further complicated academic evaluation by enabling rapid manuscript production, potentially inflating publication counts without corresponding research depth. This paper introduces a weighted algorithm that adjusts the h-index by assigning lower weights to LTEs and Correspondences, based on predefined parameters. Using example datasets, the adjusted h-index significantly reduced scores compared to the original, reflecting more accurate merit assessments. Statistical analysis revealed a high correlation between original and adjusted h-indices (Pearson r = 0.98, Spearman r = 0.94), yet highlighted a weak correlation between letter contributions and rank order (r = -0.04). Paired t-tests confirmed significant differences between the two indices (p = 0.0393). The proposed method effectively penalizes superficial publications while preserving the value of full-length research articles. This approach offers a rigorous and mathematically grounded framework for evaluating academic productivity, particularly in high-stakes settings like Italy’s National Scientific Qualification competitions. Furthermore, integrating AI tools into evaluation systems could enhance transparency and fairness, reducing biases and legal disputes related to subjective judgments. Future research should refine these metrics for broader disciplinary applications, ensuring a balanced assessment of academic contributions in the era of artificial intelligence and automated publishing tools.
Declarations
Conflict of Interest
The Author declares that there is no conflict of interest.
Author Contributions
Conceptualization, Data curation, Software, Manuscript writing, reviewing and submitting.
References
- Chirumbolo S, Bjørklund G. Pressure to publish in the biomedical scientific field: Ethical conflicts or a possible obsessive-compulsive disorder? Eur J Intern Med. 2018 Apr;50:e16-e17.
- Chirumbolo S, Bjørklund G. Scientific Pressure to Publish: An OCD or a Gambling Disorder-Like Addictive Behaviour?[Presión científica para publicar: ¿un TOC o un comportamiento adictivo similar al del juego?] Rev Colomb Psiquatria, 2022, doi: 10.1016/j.rcp.2022.10.010
