QUALITY ESTIMATION OF LEARNING OBJECTS IN REPOSITORIES OF OPEN EDUCATIONAL RESOURCES BASED ON STUDENT INTERACTIONS
DOI:
https://doi.org/10.5944/educxx1.20196Keywords:
Educational resources, learning objects, repository, educational quality, learning analytics, interaction.Abstract
Open educational resources have emerged as one of the cornerstones ofopen education. One of the main barriers hampering their use and adoption is the lack of sustainable and effective quality control mechanisms in digital repositories. Evaluation strategies such as peer review have not been sufficiently scalable to keep up with the fast pace of open content creation by the user community. This study presents a new approach grounded on learning analytics in order to estimate the quality of learning objects based on the interactions that students have with them in open environments. For the study, 146291 sessions of student interactions with 256 learning objects distributed through an open repository were analyzed. A total of 11 studentlearning object interactions were considered in the study. The quality of the resources was measured using the standard evaluation instrument LORI (Learning Object Review Instrument). To study the relationships between the student interactions with the learning objects and their quality as well as to build a predictive metric, linear regression analyses were used. The results show that there is a relationship between interactions and quality, and that it is possible to estimate with a moderate error the quality of the learning objects based on the interactions that students have with them. The results obtained point out that the proposed learning analytic can be used in open learning object repositories to automatically detect conflicting or low quality resources.
Downloads
References
Agudo-Peregrina, Á. F., Iglesias-Pradas, S., Conde-González, M.Á., & Hernández-García, Á. (2014). Can we predict success from log data in VLEs? Classification of interactions for learning analytics and their relation with performance in VLE-supported F2F and online learning. Computers in Human Behavior, 31(1), 542–550. doi:10.1016/j.chb.2013.05.031
Baki, A., & Çakıroğlu, Ü. (2010). Learning objects in high school mathematics classrooms: Implementation and evaluation. Computers & Education, 55(4), 1459–1469. doi:10.1016/j.compedu. 2010.06.009
Baraniuk, R. G. (2008). Challenges and opportunities for the Open Education Movement: A Connexions Case Study. In Opening Up Education: The Collective Advancement of Education through Open Technology, Open Content, and Open Knowledge (pp. 229–246). MIT Press.
Barra, E., Gordillo, A., & Quemada, J. (2014). Virtual Science Hub: An Open Source Platform to Enrich Science Teaching. International Journal of Social, Behavioral, Educational, Economic, Business and Industrial Engineering, 8(3), 741 – 746.
Chitwood, K. (2005). Learning Objects: Making a Difference in Teaching and Learning. In 20th Annual Conference on Distance Teaching and Learning (pp. 1–5). Wisconsin Online Resource Center.
Clements, K., & Pawlowski, J. (2012). User-oriented quality for OER: understanding teachers’ views on re-use, quality, and trust. Journal of Computer Assisted Learning, 28(1), 4–14. doi:10.1111/j.1365-2729.2011.00450.x
Clements, K., Pawlowski, J., & Manouselis, N. (2015). Open educational resources repositories literature review – Towards a comprehensive quality approaches framework. Computers in Human Behavior, 51, part B, 1098–1106. doi:10.1016/j.chb.2015.03.026
Clow, D. (2012). The learning analytics cycle: closing the loop effectively. In Proceedings of the 2nd international conference on Learning Analytics and Knowledge (LAK) (pp. 134–138). doi:10.1145/2330601.2330636
Downes, S. (2007). Models for Sustainable Open Educational Resources. Interdisciplinary Journal of Knowledge and Learning Objects, 3, 29-44.
Dyckhoff, A. L., Zielke, D., Bültmann, M., Chatti, M. A., & Schroeder, U. (2012). Design and Implementation of a Learning Analytics Toolkit for Teachers. Educational Technology & Society, 15(3), 58–76.
Gómez-Zermeño, M. (2012). Bibliotecas digitales: recursos bibliográficos electrónicos en educación básica. Comunicar, 20(39), 119–128. doi:10.3916/C39-2012-03-02
Gordillo, A., Barra, E., & Quemada, J. (2014). Towards a Learning Object pedagogical quality metric based on the LORI evaluation model. In Proceedings of the 2014 Frontiers in Education Conference (FIE 2014). doi:10.1109/FIE.2014.7044499
Hanley, G. (2005). Enabling Open Education with MERLOT. In Proceedings of the Advancing the Effectiveness and Sustainability of Open Education Conference (OpenEd 2005). Recuperado de http://tinyurl.com/opened2005
Hung, J. L., Hsu, Y. C., & Rice, K. (2012). Integrating Data Mining in Program Evaluation of K-12 Online Education. Educational Technology & Society, 15(3), 27–41.
Hylén, J. (2006). Open Educational Resources: Opportunities and Challenges. In Proceedings of the 2006 Open Education Conference (OpenEd 2006) (pp. 49–63). Recuperado de https://goo.gl/ZbtJPe
Kay, R. (2012). Exploring the use of web-based learning tools in secondary school classrooms. Interactive Learning Environments, 22(1), 67–83. doi: 10.1080/10494820.2011.641675
Kay, R., Knaack, L., & Petrarca, D. (2009). Exploring Teachers Perceptions of Web-Based Learning Tools. Interdisciplinary Journal of E-Learning and Learning Objects, 5, 27–50.
Krauss, F., & Ally, M. (2005). A Study of the Design and Evaluation of a Learning Object and Implications for Content Development. Interdisciplinary Journal of E-Learning and Learning Objects, 1, 1–22.
Larsen, K., & Vincent-Lancrin, S. (2006). The impact of ICT on tertiary education: advances and promises. In B. Kahin & D. Foray (Eds.), Advancing Knowledge and the Knowledge Economy (pp. 151–168). Boston (MA): MIT Press.
Leacock, T. L., & Nesbit, J. C. (2007). A Framework for Evaluating the Quality of Multimedia Learning Resources. Educational Technology & Society, 10(2), 44–59.
Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an "early warning system" for educators: A proof of concept. Computers and Education, 54(2), 588–599. doi:10.1016/j.compedu.2009.09.008
Marcus-Quinn, A., & McGarr, O. (2015). El uso de los objetos de aprendizaje reutilizables en la enseñanza de la poesía inglesa: explorando la influencia de las prácticas pedagógicas predominantes. Educación XX1, 18(1), 325–344. doi:10.5944/educxx1.18.1.12335
McDonald, J. (2006). Learning object: A new definition, a case study and an argument for change. In Proceedings of the 23rd annual ascilite conference: Who’s learning? Whose technology? (pp. 535–544). Recuperado de https://goo.gl/0Ez6Wx
McGreal, R. (2004). Learning Objects: A Practical Definition. International Journal of Instructional Technology and Distance Learning, 1(9), 21–32.
Miller, L. D., & Soh, L. K. (2013). Significant Predictors of Learning from Student Interactions with Online Learning Objects. In Proceedings of the 2013 Frontiers in Education Conference (FIE 2013) (pp. 203–209). doi:10.1109/FIE.2013.6684817
Minaei-Bidgoli, B., Kashy, D. A., Kortemeyer, G., & Punch, W. F. (2003). Predicting student performance: an application of data mining methods with an educational Web-based system. In Proceedings of the 2003 Frontiers in Education Conference (FIE 2003). doi:10.1109/FIE.2003.1263284
Moore, M. G. (1989). Editorial: Three types of interaction. American Journal of Distance Education, 3(2), 1–7. doi:10.1080/08923648909526659
Muñoz-Merino, P. J., Ruipérez-Valiente, J. A., Alario-Hoyos, C., Pérez-Sanagustín, M., & Kloos, C. D. (2014). Analítica del Aprendizaje para la Evaluación Precisa de la Efectividad del Alumno con Actividades y Recursos Educativos. In Proceedings of the 9th Iberian Conference on Information Systems and Technologies (CISTI 2014) (pp. 1 – 6). doi:10.1109/CISTI.2014.6877096
Ochoa, X., & Duval, E. (2006). Use of Contextualized Attention Metadata for Ranking and Recommending Learning Objects. In Proceedings of the 1st international workshop on Contextualized attention metadata: collecting, managing and exploiting of rich usage information. doi:10.1145/1183604.1183608
Osorio, L. A., & Duart, J. M. (2011). Análisis de la interacción en ambientes híbridos de aprendizaje. Comunicar, 35, 65–72. doi:10.3916/C37-2011-02-06
Palazón, A. (2001). Comunicación web: el valor de los contenidos de la Red. Comunicar, 17, 93–96.
Romero, C., & Ventura, S. (2010). Educational Data Mining: A Review of the State of the Art. IEEE Transactions on Systems, Man, and Cybernetics- Part C: Applications and Reviews, 40(6), 601–618. doi:10.1109/TSMCC.2010.2053532
Schibeci, R., Lake, D., Phillips, R., Lowe, K., Cummings, R., & Miller, E. (2008). Evaluating the use of learning objects in Australian and New Zealand schools. Computers & Education, 50(1), 271–283. doi:10.1016/j.compedu.2006.05.006
Siemens, G., & Gasevic, D. (2012). Guest Editorial - Learning and Knowledge Analytics. Educational Technology & Society, 15(3), 1–2.
Long, P., & Siemens, G. (2011). Penetrating the Fog: Analytics in Learning and Education. EDUCAUSE Review, 46(5), 31-40.
Su, J. M., Tseng, S. S., Wang, W., Weng, J. F., Yang, J. T. D., & Tsai, W. N. (2006). Learning Portfolio Analysis and Mining for SCORM Compliant Environment. Educational Technology & Society, 9(1), 262–275.
Tzikopoulos, A., Manouselis, N., & Vuorikari, R. (2007). An Overview of Learning Object Repositories. In Learning Objects for Instruction: Design and Evaluation (pp. 29–55). doi:10.4018/978-1-59904-334-0.ch003
Van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2014). Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL. Computers and Education, 79, 28–39. doi:10.1016/j.compedu.2014.07.007
Vargo, J., Nesbit, J. C., Belfer, K., & Archambault, A. (2003). Learning Object Evaluation: Computer-Mediated Collaboration And Inter-Rater Reliability. International Journal of Computers and Applications, 25(3), 198–205.
Wiley, D. A. (2000a). Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. In The Instructional Use of Learning Objects: Online Version. Recuperado de http://tinyurl.com/wileyconnecting
— (2000b). Learning Object Design and Sequencing Theory. Brigham Young University.
William and Flora Hewlett Foundation. (2013). Open Educational Resources: Breaking the Lockbox on Education (White Paper). Recuperado de https://goo.gl/vmPk7a
Zervas, P., Alifragkis, C., & Sampson, D. G. (2014). A quantitative analysis of learning object repositories as knowledge management systems. Knowledge Management & E-Learning, 6(2), 156–170.
Downloads
How to Cite
Issue
Section
License
Educación XX1 is published under a Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC 4.0)
