Achievements and challenges in learning analytics in Spain: The view of SNOLA

As in other research fields, the development of learning analytics is influenced by the networks of researchers that contribute to it. This paper describes one of such networks: the Spanish Network of Learning Analytics (SNOLA). The paper presents the research lines of the members of SNOLA, as well as the main challenges that learning analytics has to address in the next few years as perceived by these researchers. This analysis is based on SNOLA’s archival data and on a survey carried out to the current members of the network. Although this approach does not cover all the activity related to learning analytics in Spain, the results provide a representative overview of the current state of research related to learning analytics in this context. The paper describes these trends and the main challenges, among How to reference this article: Martínez-Monés, A., Dimitriadis, Y., Acquila-Natale, E., Álvarez, A., Caerio-Rodríguez, M., Cobos, R., Conde-González, M. A., García-Peñalvo, F. J., Hernández-Leo, D., Menchaca, I., Muñoz-Merino, P. J., Ros, S., y Sancho-Vinuesa, T. (2020). Achievements and challenges in learning analytics in Spain: The view of SNOLA. RIED. Revista Iberoamericana de Educación a Distancia, 23(2), (preprint). doi: http://dx.doi.org/10.5944/ried.23.2.26541 A. MARTÍNEZ-MONÉS; Y. DIMITRIADIS; E. ACQUILA-NATALE; A. ÁLVAREZ; M. CAERIO-RODRÍGUEZ; R. COBOS; M. A. CONDEGONZÁLEZ; F. J. GARCÍA-PEÑALVO; D. HERNÁNDEZ-LEO; I. MENCHACA; P. J. MUÑOZ-MERINO; S. ROS; T. SANCHO-VINUESA ACHIEVEMENTS AND CHALLENGES IN LEARNING ANALYTICS IN SPAIN: THE VIEW OF SNOLA RIED. Revista Iberoamericana de Educación a Distancia (2020), 23(2), (versión Preprint) DOI: http://dx.doi.org/10.5944/ried.23.2.26541 – ISSN: 1138-2783 – E-ISSN: 1390-3306 which we can point out the need to adopt an ethical commitment with data, to develop systems that respond to the requirements of the end users, and to reach a wider institutional impact.

Research in Information Science, as well as in many other disciplines, is enacted by intertwined networks of people and technology (Latour, 2005). Learning analytics (LA) as a research field illustrates this idea well, as it has witnessed a fast development, partially due to the intense work of different networks of researchers that have sustained it. The society for Learning Analytics Research (SOLAR) (Solar, 2020) is the most influential, bringing together researchers and actors related to LA across the globe. While creating SOLAR and with the aim of fostering research on LA at more local levels, several networks were founded at the beginning of the 2010's in different regions. Some of them started out as informal organizations of people, whereas in some cases they were created on the basis of a founded project. One example of these officially recognized networks at the European level is the Learning Analytics Community Exchange (LACE) (LACE, 2020), which contributed with some outstanding work between the years 2015 and 2017 and is still active in connection with SOLAR.
In Spain a group of researchers working in the LA field formed the Spanish Network of Learning Analytics (SNOLA) (SNOLA, 2020) in 2013. This network was officially recognized in 2015 by the Spanish Government. In 2020 SNOLA has started a new period of activity as an officially recognized thematic network, with new groups that add more variety and quality to the network. Although SNOLA does not represent all the work being done in LA in Spain, it comprises a wide representation of groups. The description of the work done by its members provides a good picture of what is being done in LA research in Spain, in line with recent works providing a more general overview of the field (Papamitsiou, Giannakos, & Ochoa, 2020).
Scientific networks on specific areas can be developed at different levels, e.g. global/regional/national, or coming from other existing networks and societies. The analysis and study of the networks at different levels, origins or focuses may be relevant in order to understand the trends and challenges that are in the center of attention of these networks. This is especially relevant for the field of LA that emerged in the last ten years as a consequence of the interest of a number of researchers that were working in related research areas (i.e., Artificial Intelligence in Education, Computer Supported Collaborative Learning, Educational Data Mining, or Technology Enhanced Learning), each one of them with its own community behind. Although some studies have been undertaken in order to provide an overview of the global landscape in LA (Ferguson et al., 2016), it is still important to further analyze and understand the evolution, trends and challenges that are detected in an active network at a finer grain level (national). This paper presents SNOLA: its motivation, goals, activities, and expected outcomes in the following years. It provides an overview of the work carried out by its members, giving a partial but meaningful panorama of current research and challenges of LA in Spain. This overview is based on archival information about the activity of the network in the past years and its working plan for the future. As a second contribution, the paper presents an outline of the challenges that should be faced by the LA research field in the next few years. The description of the activity of the groups and their vision of the challenges was collected through an open-ended survey that was filled out by the representative of each group in the network. Using this survey, we have collected the main research lines of each group related to LA, as well as the participants' view of the main challenges of the LA research field in the near future.
The structure of the paper is as follows. First, we describe SNOLA, its motivation and main goals, the activities it has carried out in the past and what impact is expected from the network in the future. Then, the research groups belonging to SNOLA are introduced. Third, we describe the work done by its members in the last 5 years, followed by a summary and discussion of the main challenges for LA described by its members.

SNOLA -MOTIVATION AND ACTIVITIES
Education is a key element in our society. Institutional reports by the European Commission (EC, 2016) have pointed out the potential of LA for the improvement of the quality of the European education system, but also the fact that many research questions and technical issues need to be solved to reach this goal. The need to face these issues has led the European Union to make a series of recommendations for the field of LA (Ferguson et al., 2016). One of these recommendations is the need for structured organizations to support the use of LA; the urgency of bringing together all relevant stakeholders to collect and analyze the evidence about the present and expected benefits of LA; and to identify the needs of the implied actors, especially students and teachers.
SNOLA appeared with the main aim to face these demands. It was established informally in 2013 by Spanish researchers interested in the interchange of knowledge and expertise in the area of LA. Its mission was to create synergies and explore possibilities of collaboration among them, as well as becoming a reference in the field of LA in Spain. SNOLA also aimed to promote the connection between Spanish researchers with other international networks in different regions of the world, including Latin America, Australia, US and Europe. The work of the network was reinforced by its official recognition as a Thematic Network of Excellence by the Spanish Government in 2015 and in 2020.
One of the main activities of the network since its creation has been the organization of scientific events that have served to disseminate the work of the network and to extend the connections to other researchers, both in Spain and at an international level. Since 2013, SNOLA has organized numerous events: the series of Learning Analytics Summer Schools, (LASI Spain) which have taken place annually since its first edition in 2013, in Madrid, up to its 2019 edition in Vigo (Caeiro-Rodríguez, Hernández-García, & Muñoz-Merino, 2019).
These networking activities have helped to extend the network to other forums, such as the events organized by the e-Madrid network, the organization of the special track on LA in the TEEM conference from 2013 (Conde-González & Hernández-García, 2013) to 2019 (Conde-González & Hernández-García, 2019), and the organization of the workshops WLA-CISTI and WAPLA. This intense activity has been possible due to the existence of the network itself. The communication and coordination channels set up by the SNOLA have facilitated to build up working teams to organize these events in a smooth and efficient fashion, which would have been not possible without the existence of the network.
The network has maintained contact with relevant researchers in the field of LA at the international level, who have offered webinars organized by the network, or have participated as keynote speakers at different editions of LASI and the special tracks on LA organized by SNOLA. These connections have materialized as well in the participation of some of the members of the network in international projects. For example, in Europe, the project SHEILA (Tsai et al., 2018) has worked in the deployment of LA policies at an institutional level. This project has had a continuation in Latin America with the project of Learning Analytics in Latin America (LALA, 2020) Finally, it is interesting to note that the number of organizations (companies, public administrations, universities, educational institutions in general) and persons interested in the network goes beyond the list of the official members of SNOLA. The SNOLA webpage (SNOLA, 2020) has become a hub for information about LA in Spain, with news, resources, software, and publications that provide a good overview of the current state of research in the area in our context.

SNOLA COMPONENTS
Besides the networking and dissemination activities, the value and richness of the network is derived from the activity of its members. This section describes the groups that belong to SNOLA and the main research lines in which they are involved.
SNOLA, as an officially recognized thematic network, comprises 12 research groups, all of them belonging to different universities across Spain. These groups are summarized in Table 1, which includes a link to their URL. As it can be observed, most of its members come from a technological background, from pure Information Technology (IT)-related areas like Telematics Engineering, Computer Languages and Systems, Artificial Intelligence, and Computer Architecture; or applied areas, such as Didactics of Mathematics or Business Organization. Therefore, SNOLA as a whole has been traditionally more focused on technology-oriented contributions rather than other aspects which are also important (e.g. pedagogical, philosophical, etc.). This has been also the case with the wider LA field at an international level. However, the community is fully aware of the need to consider the ethical, social, and pedagogical issues raised by the domain in which these technological contributions are presented. The increasing weight of these issues in current discussions in the field points out to the need of a more interdisciplinary approach in the composition of the groups and of the network. A challenge of SNOLA is to attract researchers that come from social and pedagogical fields to collaborate closely with its members, in order to face the more global challenges posed nowadays in LA. These challenges are discussed more deeply in the following section. Table 1 evidences a considerable experience in LA of the members of SNOLA, given the range of years in the field (from 5 to 20 years). Some of these groups were already working in research problems that are now considered as part of LA A. MArtínez-Monés; Y. DiMitriADis; e. AcquilA-nAtAle; A. ÁlvArez; M. cAerio-roDríguez; r. cobos; M. A. conDe-gonzÁlez; F. J. gArcíA-PeñAlvo; D. HernÁnDez-leo; i. MencHAcA; P. J. Muñoz-Merino; s. ros; t. sAncHo-vinuesA AcHieveMents AnD cHAllenges in leArning AnAlYtics in sPAin: tHe view oF snolA years before the field was recognized as a separate research area with this name. This is explained by the fact that most of the groups that belong to SNOLA joined the specific LA field from a broader research field related to the use of computers in education, like e-learning, Technology Enhanced Learning (TEL), Computer Supported Collaborative Learning (CSCL), Educational Data Mining (EDM) or Artificial Intelligence in Education (AIED). * For each member, the name of the Group, its URL, the knowledge area of its members, the university to which they belong, and the number of years working in LA and related areas previous to 2011

SNOLA LINES OF RESEARCH
In order to provide an overview of the work done by the network, the groups reported their research lines in an internal survey, which was inspired by the classification of LA systems presented in Omedes (2018). A total of 34 distinct research lines were identified. For each research line, the following data has been collected: brief description; who is the main user of the system/functionality; educational level to which it has been applied/thought of; data sources and analysis techniques; the type of system (whether it is descriptive or prescriptive); and the A. MArtínez-Monés; Y. DiMitriADis; e. AcquilA-nAtAle; A. ÁlvArez; M. cAerio-roDríguez; r. cobos; M. A. conDe-gonzÁlez; F. J. gArcíA-PeñAlvo; D. HernÁnDez-leo; i. MencHAcA; P. J. Muñoz-Merino; s. ros; t. sAncHo-vinuesA AcHieveMents AnD cHAllenges in leArning AnAlYtics in sPAin: tHe view oF snolA main expected benefits of the system. Regarding this feature, we started with the four possible values suggested by Omedes (2018), and added three additional ones that emerged from the data provided by the partners. This set of categories are shown in Table 2, with a description of their meaning, the code we will use to refer to them in this section, and the number of research lines that were reported to fall under each research goal. 3 * Categories (and codes) used to describe the different research goals of a learning analytics system / functionality and how many times these goals were reported with a research line. The first four categories are the ones suggested by Omedes (2018). The categories in the last three rows emerged from the responses to the survey The data reported by SNOLA members has been classified according to their similarities in a number of categories. This classification is not orthogonal: some of the research lines might belong to different categories, but we present them in only one of them, to make the description more clear and simpler to follow.

Predictive LA systems
The first category is composed of predictive LA systems. Prediction of learning variables, such as students' performance or students at risk is one of the most well-known functionalities of LA applications (Peña-Ayala, 2018). It is therefore not surprising that it is the most frequently reported research line, with six lines (see Table 3). Some of these lines focus on the prediction of learning results and dropout in MOOCs (Moreno-Marcos et al., 2020;Cobos & Olmos, 2018), or on-line and blended courses (Martínez, Campuzano, Sancho-Vinuesa, & Valderrama, 2019; Agudo-Peregrina, Iglesias-Pradas, Conde-González, & Hernández-García, 2014). In general, these systems aim at deriving high-level indicators from the low-level data provided by the system logs (Alexandron, Ruipérez-Valiente, Chen, Muñoz-Merino, & Pritchard, 2017). One of such indicators is academic engagement (Bote-Lorenzo & Gómez-Sánchez, 2018). The users of these systems are students and teachers, who are expected to benefit from actionable information provided by the predictive systems. In some of the cases they also address managers in charge of the institutional learning environment, or researchers that aim at identifying indicators for learning (IIL) based on the low-level data provided by the system. In general, works related to prediction need a certain amount of data. For this reason, they are frequent in virtual learning contexts at scale, enacted on LMSs (like Moodle) or MOOCs, although other environments like version control systems are considered as well (Guerrero-Higueras, DeCastro-García, Rodríguez-Lera, Matellán, & Conde, 2019). As it can be expected, these research lines use different versions of predictive analysis techniques from artificial intelligence and machine learning. The main expected benefits from these research lines are to improve learner retention and performance (ILRP) as well as to improve the quality of the course content and the learning experience (IQCCLE).

Visual Analytics
A second category of research lines frequent in SNOLA is visual analytics (see Table 4). The focus on data visualization is also reported by Peña-Ayala (2018) as a main functionality of LA systems. In SNOLA, the groups that work in this research line aim at providing dashboards oriented to teachers (Gómez-Aguilar, García-Peñalvo, & Therón, 2014), students (Tobarra et al., 2014) or both (Ruipérez-Valiente, Muñoz-Merino, Leony & Delgado Kloos 2015). These systems are generally aimed at providing dashboards with indicators derived from the log data, but some of them are oriented to more specific purposes, like the one by Chaparro-Peláez, Iglesias-Pradas, Rodríguez-Sedano and Acquila-Natale (2019), which aims at supporting peer-and self-assessment processes. Most of these systems have a descriptive nature, although some others combine descriptive and prescriptive objectives (Tobarra et al., 2014). Sometimes these systems are also designed with the aim of helping managers allocate costs effectively (Cobos, Gil, Lareo, & Vargas, 2016), besides other outcomes, such as helping teachers identify indicators for learning. These systems use techniques specific to visual analytics, sometimes supported by heuristics or descriptive statistics. Some of the reported research lines are currently focused on frameworks to facilitate the automatic generation of visualizations, like the system by Vázquez-Ingelmo, García-Peñalvo, and Therón (2019), to generate dashboards adapted to the needs of different types of users, or the one by Hernández-García and Suárez-Navas, (2017) oriented to the creation of graph data based on social network analysis.

Support to active learning strategies
The systems in the previous categories are based on distance learning and/or MOOCs, but LA is also used to promote other types of learning. Table 5 describes the lines of research related to this goal. These types of learning include collaborative learning (Amarasinghe, Hernández-Leo, & Jonsson, 2019); adaptive learning (Muñoz-Merino, Novillo & Delgado Kloos, 2018); peer feedback (Er, Dimitriadis, & Gaseviç, 2019); social learning (Claros, Cobos, & Collazos, 2015), and flipped classrooms (Rubio-Fernández, Muñoz-Merino, & Delgado Kloos, 2019). All these proposals share their orientation for both teachers and students. In general, they are expected to improve learner retention and performance (ILRP); to proactively drive success (PDS), by analyzing the paths that make the students perform better and using this knowledge to make recommendations to the users; and at improving the quality of the course content and the learning experience (IQCCLE). In many cases, the results of the LA system are used to improve the learning design (LD) of the courses. This link between LA and LD is gaining momentum in the field of LA and may have higher impact in the upcoming years.
A slightly different approach related to this category is the one taken by Manso-Vázquez, Caeiro-Rodríguez and Llamas-Nistal, (2018) that aims at defining design criteria for self-regulated learning, including the definition of the traces that should be generated by these systems according to the xAPI specification. Table 5. Research lines related to the use of LA to support different types of online learning (see Table 3 for the description of the codes)

Analysis techniques
Type Goal

Learning analytics for Learning Design
The fifth category recognizes the importance of the relation between learning design and LA in a different direction: using LA to support LD processes. This is the common goal of the two research lines reported in Table 6. The system by Michos, Hernández-Leo and Albó (2018) uses the actions on a social network for teachers to support learning design processes, while the line reported by Wiley, Dimitriadis, Bradford and Linn (2020) aims at developing a framework for developing and evaluating learning analytics for learning design. Table 6. Research lines related to the use of LA to support learning design processes (see Table 3 for the description of the codes)

Assessment support
Assessment is a central aspect of learning. In response to this fact, some of the groups in SNOLA have devoted their work to this topic. The work by Villamañe, Larrañaga and Álvarez (2017) aims at supporting these processes by providing indicators to assist evaluators in adjusting their grades. Others have focused on supporting the assessment of 21st century skills (Menchaca Sierra, Guenaga, & Solabarrieta, 2018), some of them with a focus on workgroup assessment Conde, Colomo-Palacios, García-Peñalvo, & Larrucea, 2018; Hernández-García, Acquila-Natale, Chaparro-Peláez, & Conde, 2018). Table 7. Research lines related to assessment (see Table 3 for the description of the codes)

Analysis techniques
Type Goal

Multimodal and contextual data
One of the challenges in LA is to improve the quality of the data used to derive indicators. In this order, one category of the research works reported in this study is devoted to the definition of new data sources for LA (see Table 7). These works explore the use of human-generated data like grades (Villamañe, Alvarez, & Larrañaga, 2020) or information provided by teachers in the learning design (Rodríguez-Triana, Martínez-Monés, Asensio-Pérez, & Dimitriadis, 2015); biometric signals captured with sensors (de Arriba-Pérez, Caeiro-Rodríguez, & Santos-Gago, 2018); or multimodal data including both self-reported and sensor-based data (Vujovic & Hernández-Leo, 2019). Some of these works are exploratory, and are aimed at identifying indicators for learning based on new data sources or at defining better learner models. Table 8. Research lines related to the use of multimodal data and/or context-aware data (see Table 3 for the description of the codes)

Emotion and sentiment analysis
One of the ways to design improved learner models is the capability to model emotions. A group of research lines is devoted to this goal (see Table 9). For example, some of the works reported perform sentiment analysis on data taken from the posts of students in forums  or the actions of the students in MOOCs and MOOC contents (Cobos, Jurado, & Blázquez-Herranz, 2019). Another approach is the one taken by Ruiz, Urretavizcaya, Rodríguez and Fernández-Castro (2018) who modeled emotions based on self-reported data.
A. MArtínez-Monés; Y. DiMitriADis; e. AcquilA-nAtAle; A. ÁlvArez; M. cAerio-roDríguez; r. cobos; M. A. conDe-gonzÁlez; F. J. gArcíA-PeñAlvo; D. HernÁnDez-leo; i. MencHAcA; P. J. Muñoz-Merino; s. ros; t. sAncHo-vinuesA AcHieveMents AnD cHAllenges in leArning AnAlYtics in sPAin: tHe view oF snolA Table 9. Research lines related to emotion modelling and sentiment analysis data (see Table  3 for the description of the codes) A final research line is related to the definition of frameworks for the adoption of LA, enacted by the participation of one of the groups in the projects SHEILA (Tsai et al., 2018) and LALA. Both projects deal with one of the challenges posed to LA at a national and international level: institutional adoption of LA. The following section deals with this and other challenges for LA in the context of SNOLA.

MAIN CHALLENGES TO LA IN SPAIN FROM THE PERSPECTIVE OF SNOLA
One of the missions of SNOLA is to identify the research challenges that should drive the attention of the research and development in the field for the next few years. In the survey used for this study, the members of the network were asked to identify the main challenges faced by the LA field. A total of 36 challenges were identified by the respondents. The answers were analyzed and clustered around main topics. This section provides a brief summary of their responses. • Ethical, privacy and security issues were among the issues most frequently mentioned. Overall, the survey shows that there is a strong concern about the need for an ethical commitment with the data by the researchers working in LA. There is a common understanding that our work has to comply with the new regulations about data protection -e.g. the General Data Protection Regulation (EU, 2016)-, and that all stakeholders need more training and tools to address this need (Drarschler & Greller, 2016). This is a global challenge in LA at an international level (Slade & Prinsloo, 2013), but it has local implications, related to national-level regulations and the ways that local users (individuals, schools, universities) in our context are (or are not) developing procedures to deal with ethical and data privacy and security issues. • Partially related to the ethical commitment with data is the need to increase the quality of the analytical processes and the results. Some of the respondents pointed out that LA needs to provide valid and reliable results, so that users have the confidence to apply them in their practice. This challenge involves working in improved indicators, which provide more accurate proxies to learning, developing tools to check the validity and reliability of the LA-results, or collecting high-quality data. Some of the responses pointed out to the need of considering contextual issues to enrich the analysis and increase the validity of the results. • Besides defining better learner models, another group of challenges addressed pedagogical issues, focusing on improving real learning. This goal is at the core of the definition of LA (Gašević, Dawson and Siemens, 2015). However, its achievement has proven to be especially difficult because -as pointed out by one of the participants-LA has to deal with complex sociotechnical systems. While the general idea is to keep on doing research about learning and building improved learner models, one alternative approach to deal with the overall difficulty to address learning is suggested by another participant "to focus on providing better tools for teachers, which will lead to better learning". • Increase adoption by end users. LA solutions need to address the demands of their users while considering their constraints. This goal could be achieved A. MArtínez-Monés; Y. DiMitriADis; e. AcquilA-nAtAle; A. ÁlvArez; M. cAerio-roDríguez; r. cobos; M. A. conDe-gonzÁlez; F. J. gArcíA-PeñAlvo; D. HernÁnDez-leo; i. MencHAcA; P. J. Muñoz-Merino; s. ros; t. sAncHo-vinuesA AcHieveMents AnD cHAllenges in leArning AnAlYtics in sPAin: tHe view oF snolA by means of co-design approaches that consider the users' perspective from the outset, or by reinforcing teachers' training, thus helping to develop a culture of data-based analysis. This focus on users is shared by Ferguson et al. (2016), who pointed out that the field of LA is nowadays more represented by the "supply side" (researchers offering tools and systems) than by the "demand side" (end users), and that this should be addressed in order for LA to provide more actionable applications. • Increase personalization / interoperability of data and tools. A complementary approach to increase adoption is to focus on providing tools able to adapt / to be personalized to the needs of different types of users and contexts. In this regard, one particular issue that has to be solved is to design interoperable solutions that may enable the integration of the tools in existing educational systems and may be perceived as easier to adopt by end-users. • Take measures to apply LA at an institutional level. Finally, several participants pointed out the need for organizational structures to support the use of LA and help educational leaders implement these challenges. Some specific actions in this line would be to set up an institution, such as a center for data driven studies, or to promote initiatives around LA at institutional levels, like CRUE (Conference of the Spanish University Principals), which is the main link between the Universities and the Spanish Government. This problem of encouraging a wider institutional adoption can be addressed by considering the recommendations given by the SHEILA project (Tsai et al., 2018), in which a SNOLA member participated as a partner.

CONCLUSIONS
SNOLA has played a main role in the promotion of LA in Spain since its foundation in 2013, through the activity carried out by the network, including the organization of conferences and LA-related events, as well as the contributions of each member group to the field. The review to the research lines reported by the members of SNOLA shows a multifaceted and intense activity, covering a wide range of topics and approaches and including connections with other LA research groups and organizations at an international level.
Most of the members of the network have a technical background. As a consequence, their research lines are mostly focused on technological contributions, as described in this paper. In spite of this bias, the members of SNOLA are aware of the global challenges in the field, are not restricted to technological issues. Among these challenges, the participants highlight the need for an ethical commitment with data; the design of systems that are able to adapt to the needs and demands of their users; and the establishment of a wider institutional framework to support and foster advances in the field. In order to face these challenges, the network might benefit from establishing connections with networks, groups, or individuals coming from complementary fields, such as pedagogy, psychology and/or philosophy. These problems are aligned with the ones pointed out by Pelánek (2020) in a recent review of the challenges of the field at an international level.
This work is just a first step towards the analysis of the impact of SNOLA in the development of LA in our context, which may be complemented with new analyses (i.e., a study on the structure of dynamics of the network), or with a deeper discussion considering research in LA at an international level.