Impact of generative AI on university students’ digital competences: experimental evidence based on the DigComp framework
DOI:
https://doi.org/10.5944/ried.45533Keywords:
artificial intelligence, digital competences, higher education, autonomous learning, DigCompAbstract
This study analyzes the impact of the formative use of generative artificial intelligence (AI) on the development of digital competencies in university students. The intervention was implemented through a randomized controlled trial research design. The experimental group received training aimed at strategically using generative AI models to complete academic tasks, while the control group carried out the same activities without specific AI guidance. The impact was assessed using a difference-in-differences model with fixed effects, based on pre- and post-intervention questionnaires. Competences were analyzed according to the European DigComp 2.2 framework, covering four main competence areas: information and data literacy, communication and collaboration, safety, and problem solving. The results show statistically significant improvements in information and data literacy and in problem solving, both in their functional and metacognitive dimensions. Differential effects were also identified depending on the initial level of digital competence, with more pronounced gains among students with lower prior proficiency, who showed significant progress across all evaluated competencies. These findings suggest a compensatory effect of the didactic use of AI, capable of reducing gaps and promoting more equitable and inclusive learning processes. The study supports the guided integration of emerging technologies in higher education to strengthen digital competencies.
Downloads
References
Arseven, T., & Bal, M. (2025). Critical literacy in artificial intelligence-assisted writing instruction: A systematic review. Thinking Skills and Creativity, 57, 101850. https://doi.org/10.1016/j.tsc.2025.101850
Barak, M. (2018). Are digital natives open to change? Examining flexible thinking and resistance to change. Computers & Education, 121, 115-123. https://doi.org/10.1016/j.compedu.2018.01.016
Benvenuti, M., Cangelosi, A., Weinberger, A., Mazzoni, E., Benassi, M., Barbaresi, M., & Orsoni, M. (2023). Artificial intelligence and human behavioral development: A perspective on new skills and competences acquisition for the educational context. Computers in Human Behavior, 148, 107903. https://doi.org/10.1016/j.chb.2023.107903
Bastian, A., Liza, L. O., & Efastri, S. M. (2023). Revolutionizing education: How digital literacy is transforming inclusive classrooms in post-COVID-19. Journal of Public Health, 45(3), e609-e610. https://doi.org/10.1093/pubmed/fdad058
Blau, I., Shamir-Inbal, T., & Avdiel, O. (2020). How does the pedagogical design of a technology-enhanced collaborative academic course promote digital literacies, self-regulation, and perceived learning of students? The Internet and Higher Education, 45, 100722. https://doi.org/10.1016/j.iheduc.2019.100722
Buchholz, B. A., DeHart, J., & Moorman, G. (2020). Digital citizenship during a global pandemic: Moving beyond digital literacy. Journal of Adolescent & Adult Literacy, 64(1), 11-17. https://doi.org/10.1002/jaal.1076
Cameron, A. C., Gelbach, J. B., & Miller, D. L. (2008). Bootstrap-based improvements for inference with clustered errors. The Review of Economics and Statistics, 90(3), 414-427. https://doi.org/10.1162/rest.90.3.414
Clifford, I., Kluzer, S., Troia, S., Jakobsone, M., Zandbergs, U., Vuorikari, R., Punie, Y., Castaño-Muñoz, J., Mediavilla, C., O’Keeffe, W., & Cabrera, M. (2020). DigCompSAT: A self-reflection tool for the European Digital Competence Framework for citizens (JRC Technical Report). Publications Office of the European Union. https://doi.org/10.2760/77437
Csapó, B., & Funke, J. (Eds.). (2017). The nature of problem solving: Using research to inspire 21st century learning. OECD Publishing. https://doi.org/10.1787/9789264273955-en
Dalgıç, A., Yaşar, E., & Demir, M. (2024). ChatGPT and learning outcomes in tourism education: The role of digital literacy and individualized learning. Journal of Hospitality, Leisure, Sport & Tourism Education, 34, 100481. https://doi.org/10.1016/j.jhlste.2024.100481
Dimitriadou, E., & Lanitis, A. (2023). A critical evaluation, challenges, and future perspectives of using artificial intelligence and emerging technologies in smart classrooms. Smart Learning Environments, 10(1), Article 12. https://doi.org/10.1186/s40561-023-00231-3
Dunning, D. (2011). The Dunning–Kruger effect: On being ignorant of one’s own ignorance. In J. M. Olson & M. P. Zanna (Eds.), Advances in Experimental Social Psychology (Vol. 44, pp. 247-296). https://doi.org/10.1016/B978-0-12-385522-0.00005-6
European Commission, Directorate-General for Education, Youth, Sport and Culture (2019). Key competences for lifelong learning. Publications Office of the European Union. https://doi.org/10.2766/569540
Ferrari, A., & Punie, Y. (2013). DIGCOMP: A framework for developing and understanding digital competence in Europe. Publications Office of the European Union. https://doi.org/10.2788/52966
Gallardo-Echenique, E. E., de Oliveira, J. M., Marqués-Molias, L., Esteve-Mon, F., Wang, Y., & Baker, R. (2015). Digital competence in the knowledge society. MERLOT Journal of Online Learning and Teaching, 11(1). https://jolt.merlot.org/vol11no1/Gallardo-Echenique_0315.pdf
García Peñalvo, F. J., Llorens-Largo, F., & Vidal, J. (2024). La nueva realidad de la educación ante los avances de la inteligencia artificial generativa. RIED-Revista Iberoamericana de Educación a Distancia, 27(1), 9-39. https://doi.org/10.5944/ried.27.1.37716
Greene, W. (2004). The behaviour of the maximum likelihood estimator of limited dependent variable models in the presence of fixed effects. The Econometrics Journal, 7(1), 98-119. https://doi.org/10.1111/j.1368-423X.2004.00123.x
Güner, H., & Er, E. (2025). AI in the classroom: Exploring students’ interaction with ChatGPT in programming learning. Education and Information Technologies, 30, 12681–12707. https://doi.org/10.1007/s10639-025-13337-7
Gutiérrez Porlán, I., & Serrano Sánchez, J. L. (2016). Evaluation and development of digital competence in future primary school teachers at the University of Murcia. Journal of New Approaches in Educational Research, 5(1), 51-56. https://doi.org/10.7821/naer.2016.1.152
Hair, J. F., Jr., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7th ed.). Prentice Hall.
Hillmayr, D., Ziernwald, L., Reinhold, F., Hofer, S. I., & Reiss, K. M. (2020). The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis. Computers & Education, 153, 103897. https://doi.org/10.1016/j.compedu.2020.103897
Hwang, G. J., Xie, H., Wah, B. W., & Gašević, D. (2020). Vision, challenges, roles and research issues of artificial intelligence in education. Computers and Education: Artificial Intelligence, 1, 100001. https://doi.org/10.1016/j.caeai.2020.100001
Inamorato dos Santos, A., Chinkes, E., Carvalho, M. A. G., Solórzano, C. M. V., & Marroni, L. S. (2023). The digital competence of academics in higher education: Is the glass half empty or half full? International Journal of Educational Technology in Higher Education, 20, Article 9. https://doi.org/10.1186/s41239-022-00376-0
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121-1134. https://doi.org/10.1037/0022-3514.77.6.1121
Laanpere, M. (2019). Recommendations on assessment tools for monitoring digital literacy within UNESCO’s Digital Literacy Global Framework (Information Paper No. 56; UIS/2019/LO/IP/56). UNESCO Institute for Statistics. https://unesdoc.unesco.org/ark:/48223/pf0000366740
Laupichler, M. C., Aster, A., Schirch, J., & Raupach, T. (2022). Artificial intelligence literacy in higher and adult education: A scoping literature review. Computers and Education: Artificial Intelligence, 3, 100101. https://doi.org/10.1016/j.caeai.2022.100101
Lee, D., & Palmer, E. (2025). Prompt engineering in higher education: A systematic review to help inform curricula. International Journal of Educational Technology in Higher Education, 22, Article 7. https://doi.org/10.1186/s41239-025-00503-7
Lee, D., Arnold, M., Srivastava, A., Plastow, K., Strelan, P., Ploeckl, F., Lekkas, D., & Palmer, E. (2024). The impact of generative AI on higher education learning and teaching: A study of educators’ perspectives. Computers and Education: Artificial Intelligence, 6, 100221. https://doi.org/10.1016/j.caeai.2024.100221
Li, Y., & Ranieri, M. (2010). Are ‘digital natives’ really digitally competent?—A study on Chinese teenagers. British Journal of Educational Technology, 41(6), 1029-1042. https://doi.org/10.1111/j.1467-8535.2009.01053.x
Lin, C. J., Lee, H. Y., Wang, W. S., Huang, Y. M., & Wu, T. T. (2025). Enhancing reflective thinking in STEM education through experiential learning: The role of generative AI as a learning aid. Education and Information Technologies, 30(5), 6315-6337. https://doi.org/10.1007/s10639-024-13072-5
List, A. (2019). Defining digital literacy development: An examination of pre-service teachers’ beliefs. Computers & Education, 138, 146-158. https://doi.org/10.1016/j.compedu.2019.03.009
Mattar, J., Ramos, D. K., & Lucas, M. R. (2022). DigComp-based digital competence assessment tools: Literature review and instrument analysis. Education and Information Technologies, 27(8), 10843-10867. https://doi.org/10.1007/s10639-022-11034-3
Moravec, V., Hynek, N., Gavurová, B., & Rigelský, M. (2024). Who uses it and for what purpose? The role of digital literacy in ChatGPT adoption and utilisation. Journal of Innovation & Knowledge, 9(4), 100602. https://doi.org/10.1016/j.jik.2024.100602
Morgan, A., Sibson, R., & Jackson, D. (2022). Digital demand and digital deficit: Conceptualising digital literacy and gauging proficiency among higher education students. Journal of Higher Education Policy and Management, 44(3), 258-275. https://doi.org/10.1080/1360080X.2022.2030275
Naamati-Schneider, L., & Alt, D. (2024). Beyond digital literacy: The era of AI-powered assistants and evolving user skills. Education and Information Technologies, 29(16), 21263-21293. https://doi.org/10.1007/s10639-024-12694-z
Ng, W. (2012). Can we teach digital natives digital literacy? Computers & Education, 59(3), 1065-1078. https://doi.org/10.1016/j.compedu.2012.04.016
OECD. (2013). PISA 2012 assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. OECD Publishing. https://doi.org/10.1787/9789264190511-en
OECD. (2023). OECD Digital Education Outlook 2023: Towards an effective digital education ecosystem. OECD Publishing. https://doi.org/10.1787/c74f03de-en
Polizzi, G. (2020). Digital literacy and the national curriculum for England: Learning from how the experts engage with and evaluate online content. Computers & Education, 152, 103859. https://doi.org/10.1016/j.compedu.2020.103859
Popenici, S. A. D., & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. Research and Practice in Technology Enhanced Learning, 12(1), 22. https://doi.org/10.1186/s41039-017-0062-8
Prensky, M. (2001). Digital natives, digital immigrants. On the Horizon, 9(5), 1-6. https://doi.org/10.1108/10748120110424816
Prior, D. D., Mazanov, J., Meacheam, D., Heaslip, G., & Hanson, J. (2016). Attitude, digital literacy and self-efficacy: Flow-on effects for online learning behavior. The Internet and Higher Education, 29, 91-97. https://doi.org/10.1016/j.iheduc.2016.01.001
Rahman, M. M. (2019). 21st century skill “problem solving”: Defining the concept. Asian Journal of Interdisciplinary Research, 2(1), 64-74. https://doi.org/10.34256/ajir1917
Saklaki, A., & Gardikiotis, A. (2024). Exploring Greek students’ attitudes toward artificial intelligence: Relationships with AI ethics, media, and digital literacy. Societies, 14(12), 248. https://doi.org/10.3390/soc14120248
Selwyn, N. (2009). The digital native–myth and reality. Aslib Proceedings: New Information Perspectives, 61(4), 364-379. https://doi.org/10.1108/00012530910973776
Ting, Y.-L. (2015). Tapping into students’ digital literacy and designing negotiated learning to promote learner autonomy. The Internet and Higher Education, 26, 25-32. https://doi.org/10.1016/j.iheduc.2015.04.004
Tomczyk, Ł. (2020). Skills in the area of digital safety as a key component of digital literacy among teachers. Education and Information Technologies, 25(1), 471-486. https://doi.org/10.1007/s10639-019-09980-6
Van Laar, E., van Deursen, A. J. A. M., van Dijk, J. A. G. M., & de Haan, J. (2020). Determinants of 21st-century skills and 21st-century digital skills for workers: A systematic literature review. Sage Open, 10(1), 1-14. https://doi.org/10.1177/2158244019900176
Vuorikari, R., Kluzer, S., & Punie, Y. (2022). DigComp 2.2: The digital competence framework for citizens – with new examples of knowledge, skills and attitudes [EUR 31006 EN; JRC128415]. Publications Office of the European Union. https://doi.org/10.2760/115376
Vuorikari, R., Pokropek, A., & Castaño Muñoz, J. (2025). Enhancing digital skills assessment: Introducing compact tools for measuring digital competence. Technology, Knowledge and Learning. Advance online publication. https://doi.org/10.1007/s10758-025-09825-x
Walstad, W. B., Rebeck, K., & Butters, R. B. (2013). Test of Economic Literacy: Examiner’s manual (4th ed.). Council for Economic Education. https://www.econedlink.org/wp-content/uploads/2018/09/TEL-Manual.pdf
Walstad, W. B., Watts, M., & Rebeck, K. (2007). Test of Understanding of College Economics: Examiner’s manual (4th ed.). Council for Economic Education. https://www.econedlink.org/wp-content/uploads/2018/09/TUCE-4th.pdf
Wang, J., & Fan, W. (2025). The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: Insights from a meta-analysis. Humanities and Social Sciences Communications, 12, 621. https://doi.org/10.1057/s41599-025-04787-y
Wang, W.-S., Lin, C.-J., Lee, H.-Y., Huang, Y.-M., & Wu, T.-T. (2025). Integrating feedback mechanisms and ChatGPT for VR-based experiential learning: Impacts on reflective thinking and AIoT physical hands-on tasks. Interactive Learning Environments, 33(2), 1770-1787. https://doi.org/10.1080/10494820.2024.2375644
Webb, M. D. (2014). Reworking wild bootstrap based inference for clustered errors (QED Working Paper No. 1315). Queen’s University, Department of Economics. https://qed.econ.queensu.ca/working_papers/papers/qed_wp_1315.pdf
Zhai, X., Chu, X., Chai, C. S., Jong, M. S. Y., Istenič, A., Spector, J. M., Liu, J., Yuan, J., & Li, Y. (2021). A review of artificial intelligence (AI) in education from 2010 to 2020. Complexity, 2021, 8812542. https://doi.org/10.1155/2021/8812542
Zhao, Y., Pinto Llorente, A. M., & Sánchez Gómez, M. C. (2021). Digital competence in higher education research: A systematic literature review. Computers & Education, 168, 104212. https://doi.org/10.1016/j.compedu.2021.104212
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Concepción González García, Nina Pallarés
This work is licensed under a Creative Commons Attribution 4.0 International License.
The articles that are published in this journal are subject to the following terms:
1. The authors grant the exploitation rights of the work accepted for publication to RIED, guarantee to the journal the right to be the first publication of research understaken and permit the journal to distribute the work published under the license indicated in point 2.
2. The articles are published in the electronic edition of the journal under a Creative Commons Attribution 4.0 International (CC BY 4.0) license. You can copy and redistribute the material in any medium or format, adapt, remix, transform, and build upon the material for any purpose, even commercially. You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
3. Conditions for self-archiving. Authors are encouraged to disseminate electronically the OnlineFirst version (assessed version and accepted for publication) of its articles before publication, always with reference to its publication by RIED, favoring its circulation and dissemination earlier and with this a possible increase in its citation and reach among the academic community.
