Estudios e Investigaciones

Assessment of immersive technologies and STEM focus in initial teacher training

Valoración de tecnologías inmersivas y enfoque STEM en la formación inicial del profesorado

Francisco Silva Díaz
Universidad de Granada, Spain
Javier Carrillo Rosúa
Universidad de Granada, Spain
Gracia Fernández Ferrer
Universidad de Granada, Spain
Rafael Marfil Carmona
Universidad de Granada, Spain
Romina Narváez
Universidad de Granada, Spain

Assessment of immersive technologies and STEM focus in initial teacher training

RIED-Revista Iberoamericana de Educación a Distancia, vol. 27, núm. 1, 2024

Asociación Iberoamericana de Educación Superior a Distancia

How to cite: Silva Díaz, F., Carrillo Rosúa, J., Fernández Ferrer, G., Marfil Carmona, R., & Narváez, R. (2024). Assessment of immersive technologies and STEM focus in initial teacher training. [Valoración de tecnologías inmersivas y enfoque STEM en la formación inicial del profesorado]. RIED-Revista Iberoamericana de Educación a Distancia, 27(1), 139-162. https://doi.org/10.5944/ried.27.1.37688

Resumen: Durante los últimos años se ha detectado un progresivo interés por la integración de diversas Tecnologías Emergentes en el ámbito de la Educación, especialmente aquellas de tipo inmersivo como la Realidad Virtual Inmersiva y Realidad Aumentada. En la presente investigación se tiene por objetivo valorar la utilidad que le atribuyen los docentes en Formación Inicial en la Facultad de Ciencias de la Educación de la Universidad de Granada a este tipo de tecnologías, específicamente en el ámbito STEM, y cómo evalúan la capacidad de integrarlas en sus futuras prácticas docentes. Se utilizó una metodología mixta, donde se aplicó un cuestionario previo a toda la población (N=544) para describir las valoraciones de los participantes, seguido de un post test a una submuestra (N=58) luego de participar de una Formación Complementaria para la creación de recursos educativos inmersivos desarrollados con la plataforma CoSpaces. Los resultados revelaron una alta utilidad atribuida a las tecnologías inmersivas, destacando su potencial para mejorar la enseñanza y el aprendizaje en el ámbito STEM. Sin embargo, se identificaron desafíos relacionados con la facilidad de integración y la falta de formación adecuada en el uso de estas tecnologías. Se enfatiza la importancia de promover la capacitación docente y la alfabetización digital para aprovechar plenamente los beneficios de estas tecnologías emergentes en la educación. Se sugiere la realización de futuras investigaciones que profundicen en estrategias de formación docente y que aborden otros contextos educativos para ampliar el conocimiento sobre las implicaciones y ventajas de las tecnologías inmersivas.

Palabras clave: educación científica, tecnología de la educación, innovación pedagógica, formación de profesores, didáctica, educación STEM.

Abstract: In recent years, there has been a growing interest in the integration of various Emerging Technologies in the field of Education, especially immersive technologies such as Virtual Reality (VR) and Augmented Reality (AR). This research aims at assessing the perceived usefulness of these technologies by pre-service teachers at the Faculty of Educational Sciences of the University of Granada, specifically in the STEM field, and how they evaluate their potential for integration into their future teaching practices. A mixed-methods approach was used, including a pre-questionnaire administered to the entire population (N=544) to describe the participants' perceptions, followed by a post-test conducted with a subset (N=58) after having participated in a Complementary Training program focusing on the creation of immersive educational resources using the CoSpaces platform. The results revealed a high perceived utility of immersive technologies, highlighting their potential for enhancing teaching and learning in the STEM domain. However, challenges related to ease of integration and the lack of adequate training in the use of these technologies were identified. The importance of promoting teacher training and digital literacy to fully leverage the benefits of these emerging technologies in education is emphasized. Further research is suggested to delve into teacher training strategies and explore other educational contexts to expand the understanding of the implications and advantages of immersive technologies.

Keywords: scientific education, educational technology, educational innovation, teacher education, didactics, STEM education.

INTRODUCTION

In the field of education, particularly in the teaching of Sciences, the STEM (Science-Technology-Engineering-Mathematics) approach has gained increasing relevance in recent years. This approach focuses on promoting interdisciplinary integration of these knowledge domains to foster critical thinking, problem-solving, and creativity in students (Martín-Páez et al., 2019; Thibaut et al., 2018; Toma & Greca, 2018). Among its various goals, STEM education seeks to prepare students to face the challenges of the 21st century, where technology plays a fundamental role in society and the economy ( Bybee, 2013; Sanders, 2009). The implementation of STEM education in national and international curricula is grounded in research demonstrating the benefits of these pedagogical approaches for developing fundamental skills in students (Fleer, 2013; Toma & Meneses-Villagrá, 2019; Zollman, 2012).

As this approach strengthens, there is an increasingly recognized importance in integrating emerging technologies as an integral part of Science Education, especially within the STEM framework (Makhoka, 2017; Chng et al., 2023; Ferrada et al., 2020; Silva-Díaz et al., 2021; Xia & Zhong, 2018). The incorporation of Emerging Technologies in STEM education has demonstrated a positive impact on student learning, offering numerous benefits, including improved attitudes toward science (Aguilera & Perales-Palacios, 2018; Cabello et al., 2021; Makransky et al., 2020; Thibaut et al., 2018). Among the most relevant Emerging Technologies are Immersive Virtual Reality (IVR), Augmented Reality (AR), 3D Printing, Educational Robotics, and Sensors, just to name a few (Freeman et al., 2017; Dubé & Wen, 2022; Silva-Díaz et al., 2022).

However, the integration of technology in education has also posed new challenges for teachers (Barroso et al., 2019; Cabero-Almenara, Romero-Tena et al., 2021; Silva-Díaz et al., 2021). In many cases, educators do not feel adequately prepared or lack the necessary competencies to effectively use technological resources in the classroom. This gap between the demand for technology in the educational environment and teacher preparedness has been a subject of concern and debate (Christensen, 2002; Ertmer et al., 2012; Boel et al., 2023; Sanchez-Prieto et al., 2019). The need for developing digital teaching competence has become increasingly evident because educators must acquire skills and knowledge to get the most out of these technological tools and ensure quality teaching within the STEM context (Del Moral et al., 2022; Cabello et al., 2021). Furthermore, it is a must for teachers to develop the ability to design and create innovative technological resources to enrich and enhance their educational activities (Cabero-Almenara, Vázquez-Cano et al., 2021; Cviko et al., 2014; Del Moral et al., 2022). Therefore, teachers can adapt to the demands and challenges of the digital era, promoting more interactive and meaningful learning for their students. However, in order to achieve effective activity design that involves technological integration in STEM education, it is essential to provide teachers with opportunities for training and professional development. Of equal necessity is ensuring the access to technological resources and institutional support needed to effectively implement these tools in the classroom (Buss et al., 2018; Cabero Almenara, Romero-Tena et al., 2021; Nistor et al., 2019).

Regarding the use of Emerging Technologies as didactic resources, several studies highlight their increasing importance all over the world, especially in the context of STEM education ( Freeman et al., 2017; Dubé & Wen, 2022; Hod, 2017; Hung & Khine, 2006; Lui & Slotta, 2014).

Taking into account the insights already mentioned, a research study has been designed stating four objectives:

  1. O1. To characterize the point of view of pre-service teachers towards technology, the use of Virtual Reality (Augmented and Immersive), the ease of use of technologies for STEM learning, and the potential of technologies for learning and teaching.
  2. O2. To analyze perceptions and experiences of pre-service teachers regarding additional training for creating immersive resources with CoSpaces1.
  3. O3. To identify the benefits and challenges of integrating Emerging Technologies by delivering a seminar and the design of educational resources in the training of future STEM educators.
  4. O4. To provide recommendations for integrating Emerging Technologies into the initial teacher education program in the field of STEM education, considering the findings and results obtained in the research.

METHOD

This study uses a mixed-methods research methodology with a sequential explanatory design of two phases (Hernández Sampieri et al., 2014). In the first phase, a descriptive quantitative approach is used to analyze data collected through a questionnaire. The second phase involves a more specific analysis of a sample of respondents by using the questionnaire's six specific items that assess the ease and potential of using Virtual Laboratories (items 13 and 19), Augmented Reality (14 and 20), and Immersive Virtual Reality (15 and 21). The post-test application is conducted to the specified items because these technologies were only used during Phase 2. The difference between pre-test application to the total sample and post-test application to participant sample of the CoSpaces Immersive Resource Creation Activity (ACRI, being its acronym in Spanish) allows researchers to obtain a deeper understanding of the results obtained in the descriptive analysis. Additionally, a qualitative approach is used based on content analysis for open-ended questions that belongs to the same instrument. The purpose of mixing these approaches is to provide a comprehensive and detailed insight into the study's findings in order to provide some knowledge and understanding in the research field.

Participants

The research was conducted as part of a seminar offered to students from different undergraduate programs. The majority of the students were enrolled in courses related to the Didactics of Experimental Sciences at the University of Granada during the academic years 2020/21, 2021/22, and 2022/23. In relation to the 2020/21 and 2021/22 academic years, it is worth noting that, despite the challenges presented by the COVID-19 pandemic, the University of Granada implemented measures to ensure the continuity of in-person activities. These activities were always carried out under strict safety protocols. In particular, the Department of Didactics of Experimental Sciences decided to deliver laboratory activities in a face-to-face format, adapting laboratory and classroom capacity and making sure that safety protocols were fulfilled to protect both the students and the faculty.

In the first phase, participants were selected through intentional non-probabilistic sampling (Cardona, 2002) due to accessibility criteria. The sample consists of 554 participants who completed the questionnaire in a single application before the seminar was delivered. These participants belonged to 16 class groups (twelve class groups from 3rd year and one from 2nd year of the Primary Education degree, one class group from 2nd year, and two class groups from the Master's degree in Secondary Education, Vocational Training, and Language Teaching).

In the second phase, there is a subsample of 58 participants (62.1% female and 37.9% male, as self-identified), who selected themselves based on their interest of post-seminar autonomous work activities proposed to four class groups (three class groups from the 3rd year of the Primary Education degree – 53.4% of participants – and one class group from the Master's degree in Secondary Education, Vocational Training, and Language Teaching, specializing in Biology and Geology – 46.6% of participants). They completed the questionnaire in a second post-intervention application after completing these activities.

Data collection method

For data collection, the Emerging Technologies in STEM Education Questionnaire (CUTE-STEM, being its acronym in Spanish) was used, which was specifically developed for this study. The questionnaire consists of 27 items, with 23 close-ended questions and four dimensions for quantitative items. Seventeen questions (items 1-5 and 12-23) were rated using a 5-point Likert scale, while the remaining five items were dichotomous (items 6-11). The reliability of the questionnaire was determined by using Cronbach's alpha coefficient for all Likert-type items (17 items), obtaining an acceptable reliability (α= 0.823).

Additionally, four open-ended questions were included: a) to assess attitudes, beliefs, and knowledge related to the integration of technology in STEM education (PA_01); b) to evaluate the differences between Augmented Reality and Immersive Virtual Reality (PA_02); c) to describe the advantages and disadvantages of using Virtual Reality as a resource for STEM education (PA_03); and d) to provide a space for reflection in which students could contribute any observations they consider relevant regarding the inclusion of technology in STEM education (PA_04). Table 1 shows the distribution of questionnaire items grouped by dimensions, and their descriptions.

Table 1
Distribution of Questionnaire Items and Dimensions

Dimensions and Items Definition
A. Point of View towards technology. (α = 0.806)
1. I am interested in technology. This category examines the degree of interest in technology, their personal use of it, and their technological competence for educational purposes. It also involves assessing their critical thinking skills regarding digital content (Internet, social media, etc.).
2. I use technology for my personal leisure.
3. I use technology in my learning process.
4. I have critical thinking skills when it comes to digital content.
5. I am competent in the use of technology.
B. Use of Virtual Reality (Dichotomous Items)
6. Augmented Reality for recreational purposes. This category measures the frequency and purpose of using Virtual Reality for personal entertainment or educational activities. The goal is to assess the extent to which participants integrate technological tools and devices into their daily routines.
7. Augmented Reality for learning in a subject.
8. Augmented Reality for teaching purposes.
9. Immersive Virtual Reality for recreational purposes.
10. Immersive Virtual Reality for learning in a subject.
11. Immersive Virtual Reality for teaching purposes.
C. Ease of use of Technologies for STEM Learning (α = 0.734)
12. Ease of use of 3D Printing. This category assesses the feasibility of using Emerging Technologies as educational resources. The focus is on evaluating the practicality and potential ease of implementation of these technologies in STEM education.
13. Ease of use of Virtual Laboratories.*
14. Ease of use of Augmented Reality.*
15. Ease of use of Immersive Virtual Reality.*
16. Ease of use of Educational Robotics.
17. Ease of use of Sensors.
D. Potential of Technologies as a Resource for STEM Learning (α = 0.847)
18. Potential of 3D Printing. This category refers to the assessment of the potential use of specific technologies for teaching and learning in STEM areas.
19. Potential of Virtual Laboratories.*
20. Potential of Augmented Reality.*
21. Potential of Immersive Virtual Reality.*
22. Potential of Educational Robotics.
23. Potential of Sensors.
Source: elaborated by authors.Note: Dimension B consists of dichotomous items, so Cronbach's alpha calculation is not applicable. * Used for the post-test application (items 13, 14, 15, 19, 20, and 21).

Data Processing

The statistical analysis of the quantitative data in the research was conducted by using SPSS v26 software, while MAXQDA software, version 2020, was used for qualitative analysis.

Procedure

The research was conducted in two phases (Figure 1). The first phase involved the administration of the instrument described earlier, allowing us to characterize the students’ needs regarding the design of a training program focused on the integration of Emerging Technologies in STEM education. The initial administration of the CUTE-STEM questionnaire to different courses enabled us to make continuous improvements in the seminars. The second phase was developed based on the identified needs resulting in the design of two formative training sessions for university students.

Figure 1
Phases of Research Implementation

Phases of Research Implementation
Source: elaborated by authors.

The first of these phases corresponds to a two-hour Seminar conducted in a face-to-face format. It is aimed at providing an overview of Emerging Technologies for STEM education, with an emphasis on Augmented and Immersive Virtual Reality. Additionally, there is an optional and additional training session, in an asynchronous virtual format, offered to participants of the Seminar. In this training session the students have to design an immersive educational resource using the CoSpaces platform. The post-test was administered after completing the activity, with an estimated time frame of three weeks.

Seminar "Emerging Technologies for STEM Education" (TEC-STEM)

In the TEC-STEM Seminar, an overview of technologies, that are being implemented as learning resources, is presented. Within the technologies, didactic activities that involve the use of Immersive Virtual Reality for a learning situation are developed. Students use various Virtual Reality headsets (PlayStation VR, Oculus Go, Oculus Rift-S, Meta Quest 2, Pico Neo 3 Pro, and mobile VR headsets), which allow them to better understand their use and the integration of headsets as learning resources.

Among the activities, the RVI application "Titans of Space Plus"2 is used with Quest 2 headsets simultaneously. As part of the activity, students are asked to express their previous ideas in regard to the proportional relationships in size and distance among the planets of the Solar System by drawing. Then, they carry out the immersive experience (Figure 2).

To ensure that students understand the proper distribution of technological resources, "collaborative workstations" were set up. Each one comprises of a Quest 2 headset and an Android-based tablet (Lenovo M10). The tablets are used to project the image that students are watching while using the headsets, making it possible to know what students can see whilst using the RVI application and therefore offer to assist them quickly and easily. Additionally, it provides a solution for those students who are not wearing glasses as they "accompany" their peers on the journey through the Solar System. The activity also involves searching for relevant information for subsequent activities, which is contained within the immersive experience. This way, all team members are engaged in the process of searching for and collecting information.

Figure 2
Seminar "Emerging Technologies for STEM Education"

Seminar "Emerging Technologies for STEM Education"
Source: authors' compilation.

In order to replicate the settings used in this research, the authors recommend using an independent wireless connection system. A good solution for this is the use of mobile internet (mobile chip) and a 4G/5G wireless router.

Immersive Resource Creation Activity with CoSpaces

Regarding virtual training, the "Complementary Activity for Immersive Resources with CoSpaces" (ACRI) was implemented asynchronously, and it was offered as part of the TEC-STEM Seminar. This activity was offered voluntarily to students from the four groups participating in this training session. The main objective of this activity was to design an immersive educational resource by using the CoSpaces platform. They were asked to design a 360-degree scene that integrated various objects and elements available on the platform, to apply them to school activities in relation to teaching sciences and STEM approaches. To provide support and guidance, a detailed self-study video was developed and provided to students, outlining step-by-step how to design a scene in CoSpaces. Additionally, a video tutorial developed by the authors of this research was provided. This activity was carried out to promote the use of immersive resources and encourage the practical application of knowledge acquired in the field of experimental science education. Figure 3 provides examples of some activities developed by students.

Figure 3
Immersive Resources developed in CoSpaces

Immersive Resources developed in CoSpaces
Source: authors' compilation.

RESULTS

The results of this research are presented in relation to the phases and types of analyses conducted. In the first phase, descriptive statistics were performed to characterize the sample. In the second phase, mixed analyses were conducted. In terms of quantitative analysis, non-parametric pre-post comparison tests, using the Wilcoxon test, were applied. The sample did not meet normality criteria. Additionally, effect size was used to complement statistical analysis, evaluating Cohen's delta (d) value.

Regarding qualitative analysis, a content analysis of participants' responses was conducted to identify thematic patterns and gain a deep understanding of their experiences and perceptions.

Phase 1

Four dimensions were analyzed based on the initial administration of the CUTE-STEM questionnaire: A) Point of view towards Technology, B) Use of Virtual Reality, C) Ease of Use of Technologies for STEM Learning, and D) Potential of Technologies for Learning and Teaching.

The mean scores of the questionnaire obtained by the sample of participants in the complementary ACRI training and those who did not participate in it are presented (Figure 4). In order to determine the existence of statistical differences between both groups, the U-Mann Whitney test was applied. The results revealed that there were no significant differences except for five items.

In the item "I use technology in my learning process" (item 3), a statistically significant difference (sig = 0.016) was found between the participant and non-participant groups. The mean score of the participant group (X = 4.02) was lower than that of the non-participant group (X = 4.37), indicating that participants reported that they used technology less frequently in their learning process than the non-participant group.

In the items "Ease of use of Virtual Laboratories" (item 13), "Ease of use of Immersive Virtual Reality" (item 15), and "Ease of use of Sensors" (item 17), statistically significant differences were also found (sig = 0.001, sig = 0.007, and sig = 0.010, respectively). While participants rated items 13 and 17 favorably compared to non-participants, in the case of item 15, the non-participant group reported a better mean score.

Figure 4
Mean Scores per Likert Scale Item of the CUTE-STEAM Instrument

Mean Scores per Likert Scale Item of the CUTE-STEAM Instrument
Source: elaborated by authors.

Finally, in the item "Potential of Augmented Reality" (item 20), a statistically significant difference was found (sig = 0.009). However, in this case, the mean score of the participant group (X = 3.91) was lower than that of the non-participant group (X = 4.27), indicating that participants perceived less potential in Augmented Reality compared to the non-participant group.

These results suggest that although there are differences between both groups, these differences are slight, and they do not show a great divergence in the responses to the initial questionnaire by those who participated in the ACRI training compared to those who did not. This allows us to determine that the post-test results are likely representative of the initial sample. In the following section, the initial results for the four dimensions of the questionnaire are presented.

Point of view towards Technology (Dimension A)

High mean scores were observed in all evaluated items. Participants demonstrated a strong interest in technology (X = 4.11), indicating that they used it in their leisure time (X = 4.41) as well as in their learning process (X = 4.34). They also showed a thoughtful attitude towards digital content (X = 4.07). However, their perception of competence in using technology was slightly lower (X = 3.84) ( Table 2 and Figure 4).

Table 2
Frequency, Mean, and Standard Deviation of Items in the “Point of View towards Technology” dimension of the CUTE-STEM Instrument

N 1 2 3 4 5 X SD
1. I am interested in technology. 489 1 12 119 156 201 4.11 0.872
2. I use technology for my personal leisure. 489 1 5 55 160 268 4.41 0.744
3. I use technology in my learning process. 489 0 9 63 172 245 4.34 0.770
4. I have critical thinking skills when it comes to digital content. 489 3 14 94 213 165 4.07 0.834
5. I am competent in the use of technology. 489 2 24 148 191 124 3.84 0.875
Source: elaborated by authors.Note: Likert Scale: 1: Very Little; 2: Little; 3: Intermediate Level; 4: Much; 5: Very Much. X = Mean; SD = Standard Deviation.

Ease of Use of Technologies for STEM Learning (Dimension C)

Differences in mean scores were observed for the evaluated items. Participants perceived greater difficulty in using 3D Printing (X = 2.72). On the other hand, they considered the use of Immersive Virtual Reality (X = 2.94), Virtual Laboratories (X = 3.06), and Augmented Reality (X = 3.22) slightly easier (Table 3 and Figure 4).

Table 3
Frequency, Mean, and Standard Deviation of Items in the Ease of Use of Technologies for STEM Learning Dimension of the CUTE-STEM Instrument

N 1 2 3 4 5 Na X SD
12. Ease of use of 3D Printing. 544 50 168 170 88 21 47 2.72 1.006
13. Ease of use of Virtual Laboratories. 544 33 124 171 132 45 39 3.06 1.060
14. Ease of use of Augmented Reality. 544 24 96 185 140 56 43 3.22 1.032
15. Ease of use of Immersive Virtual Reality. 544 42 122 168 118 29 65 2.94 1.045
16. Ease of use of Educational Robotics. 544 68 136 157 123 34 26 2.84 1.125
17. Ease of use of Sensors. 544 43 131 179 115 22 54 2.88 1.010
Source: elaborated by authors.Note: Likert Scale: 1: Very Difficult; 2: Difficult; 3: Intermediate Difficulty, Neither Easy nor Difficult; 4: Easy; 5: Very Easy. X = Mean; SD = Standard Deviation. Na = Does Not Know / Did Not Answer.

Potential of Technology for STEM Learning (Dimension D)

High average scores were obtained in all the evaluated items. Participants recognized the high potential of Sensors (X = 4.03), 3D Printing (X = 4.04), Virtual Laboratories (X = 4.21), Augmented Reality (X = 4.23), and, above all, Immersive Virtual Reality (X = 4.31) and Educational Robotics (X = 4.34) as valuable resources for STEM learning (Table 4 and Figure 4).

Table 4
Frequency, mean, and standard deviation of items in the Potential of Technology for STEM Learning dimension of the CUTE-STEM instrument

N 1 2 3 4 5 Na X SD
18. Potential of 3D Printing. 544 5 17 117 202 186 17 4.04 0.888
19. Potential of Virtual Laboratories. 544 4 13 84 193 231 19 4.21 0.852
20. Potential of Augmented Reality. 544 1 13 74 208 220 28 4.23 0.799
21. Potential of Immersive Virtual Reality. 544 1 12 61 178 243 49 4.31 0.795
22. Potential of Educational Robotics. 544 4 10 48 192 252 38 4.34 0.793
23. Potential of Sensors. 544 2 25 99 187 167 64 4.03 0.895
Source: elaborated by authors.Note: Likert scale: 1: Not useful at all; 2: Slightly useful; 3: Somewhat useful; 4: Quite useful; 5: Very useful. X = Mean; SD = Standard Deviation. Na= Don't know / No answer.

Use of Virtual Reality (Dimension B)

In terms of the dichotomous items, related to Augmented Reality, a considerable participant percentage (43.6%) indicated that they have used it for recreational purposes, while a much smaller proportion (19.7%) has used it as a learning tool for a specific subject. Additionally, only a small participant number (12.1%) has used Augmented Reality for teaching purposes in their role as teachers.

Regarding Immersive Virtual Reality (IVR), a similar frequency of use is observed compared to Augmented Reality. Approximately one-third of the participants (32.4%) have used Immersive Virtual Reality for recreational purposes, while a much smaller percentage (12.7%) have used it as a learning resource in a subject. In relation to the use of Immersive Virtual Reality for teaching purposes, the frequency is even lower, with only 7.7% of participants declaring they have used it in their role as teachers (Table 5 and Figure 5).

Figure 5
Evolution of the Frequency of Use for Augmented and Immersive Virtual Reality for each Year

Evolution of the Frequency of Use for Augmented and Immersive Virtual Reality for each Year
Source: elaborated by authors. Note: Values expressed in percentage

An examination of the responses related to the use of Augmented and Immersive Virtual Reality that were divided into the year of application of the instrument, reveal that no clear trend of evolution over time is shown, regardless of the fact that ratings may vary to a certain extent. Additionally, a "sawtooth" distribution is observed with peaks in the use of Augmented Reality and Immersive Virtual Reality in recreational contexts, but with a lower evaluation in educational contexts.

Table 5

Frequency and standard deviation of the items in the Virtual Reality Use dimension of the CUTE-STEM instrument

Table 5
Frequency and standard deviation of the items in the Virtual Reality Use dimension of the CUTE-STEM instrument

N No X SD
6. Augmented Reality for recreational purposes. 544 237 307 1.56 0.496
7. Augmented Reality for learning in a subject. 544 107 437 1.80 0.398
8. Augmented Reality for teaching purposes. 544 66 478 1.88 0.327
9. Immersive Virtual Reality for recreational purposes. 544 176 368 1.68 0.468
10. Immersive Virtual Reality for learning in a subject. 544 69 475 1.87 0.333
11. Immersive Virtual Reality for teaching purposes. 544 42 502 1.92 0.267
Source: elaborated by authors.Note: X = Mean; SD = Standard Deviation.

The results presented suggest a positive attitude and a favorable perception towards the use of technology in STEM education. However, areas for improvement were also identified, such as the perception of competence in using technology and the difficulty that participants experienced in using some technologies. Regarding technologies assessed in this research project, it is clear that their application in the educational field is still limited. Although, this study revealed that participants were more familiar with Augmented and Immersive Virtual Reality and that they also had more experience using them in recreational contexts, it is evident that much effort is needed to promote the use of these technologies as teaching and learning tools, both in specific subjects and in teaching practice.

Phase 2

The results obtained in the items that assess the ease of use of Virtual Laboratories, Augmented Reality, and Immersive Virtual Reality (items 13, 14, and 15) are presented (Table 6). Likewise, the items that evaluate the potential of these technologies as resources for teaching and learning in the STEM field in Primary/Secondary Education (items 19, 20, and 21) are shown (Table 6). Due to the data collection method being optional, there is variability in the sample. Additionally, in the case of items 14 and 20, which were specifically focused on the evaluation of Augmented Reality, it is stated that only one group designed activities based on this technology, so the post-test instrument application was limited to that group.

Table 6
Results of the Wilcoxon test and effect sizes for CUTE-STEM items pretest and post-test

Pretest Post-test
N X Min Max Mo N X Min Max Mo Sig d
item 13 56 195 3.5 1 5 4 56 195 3.5 2 5 4 0.906 0
item 14 25 67 2.7 1 5 2 27 100 3.7 1 5 4 0.005** 0.975
item 15 51 130 2.6 1 5 2 58 176 3.0 1 5 3 0.019* 0.461
item 19 58 240 4.1 1 5 5 57 257 4.5 3 5 5 0.024* 0.460
item 20 27 99 3.7 1 5 4 26 101 3.9 1 5 4 0.302 0.199
item 21 53 222 4.2 1 5 5 58 247 4.3 1 5 5 0.742 0.068
Source: elaborated by authors.Note: N = participants; ∑ = sum; X = mean; Min = Minimum; Max: Maximum; Mo = Mode; Sig = Bilateral significance (0.05); d = Cohen's delta.

Regarding the results, in item 13 (virtual laboratories), results for pre and post-tests with a sample of 56 participants were obtained. The pretest mean score (X = 3.5) indicates a perception of intermediate difficulty in using virtual laboratories, while in the post-test, the same value is obtained (X = 3.5), indicating consistency in the perception of difficulty. Both the Wilcoxon test (p = 0.906) and the Effect Size (d = 0) did not reveal significant differences between pre and post scores.

In relation to item 14, which assessed Augmented Reality, there were 25 participants in the pretest and 27 in the post-test. The pretest mean suggests a certain perception of difficulty (X = 2.7), while the post-test mean (X = 3.7) shows a significant improvement in the perception of ease. The Wilcoxon test revealed significant differences between pre and post scores (p = 0.005), with a large Effect Size (d = 0.975), indicating a substantial improvement in the perception of ease of use of Augmented Reality as a resource for STEM learning and teaching in Primary/Secondary Education.

Regarding item 15, which evaluated IVR, there were 51 participants in the pretest and 58 in the post-test. The pretest mean (X = 2.6) indicates a perception of difficulty, while the post-test mean (X = 3.0) reflects a slight improvement in ease of use. The Wilcoxon test revealed significant differences between pre and post scores (p = 0.019), with a moderate Effect Size (d = 0.461), indicating a statistically significant improvement in the perception of ease of use for IVR.

On the other hand, regarding the potential of these technologies (items 19, 20, and 21), high mean scores were observed both in the pre and post-tests.

The results reveal several important findings about participants' perception of the use of Virtual Reality (Augmented and Immersive) in educational settings. Participants assessed positively the potential of these technologies in both the pretest and post-test, demonstrating recognition of their educational and learning possibilities. These high mean scores on items that evaluate the potential of technologies (items 19, 20, and 21) support the idea that participants perceive their relevance and value in educational contexts.

Differences in the perception of the ease of use of emerging technologies were observed. Regarding the use of virtual laboratories (item 13), the results indicated a perception of intermediate difficulty both in the pretest and post-test, where no significant differences were found. This suggests that the perception of difficulty remained constant over time, indicating some need to address aspects related to the accessibility and usability of virtual laboratories in educational contexts.

However, a significant improvement in the perception of ease of use was observed in the post-test compared to the pretest regarding Augmented Reality (item 14) and Immersive Virtual Reality (item 15). These results indicate that the complementary training (TEC-STEM + ACRI) had a positive impact on participants' perception of the ease of use of these technologies. The presence of significant differences between pre and post scores, based on substantial effect sizes, emphasizes the importance of providing appropriate training to promote the adoption and effective use of these emerging technologies in educational contexts.

Content Analysis (Open-Ended Questions)

The qualitative evaluation of the questionnaire was conducted through inductive coding of responses (N=58) to understand reality from the participants' perspectives and discover new insights from them. Figure 6 presents the results of the open question (PA_03): In your opinion, what advantages or disadvantages do you think the use of Immersive Virtual Reality can have in teaching Sciences in Primary/Secondary Education?

In general, the responses reflect a positive perception of Immersive Virtual Reality in educational contexts. For example, some responses indicate that "the use of Immersive Virtual Reality significantly increases student motivation and engagement, so learning sciences would be easier." (P_41). Furthermore, it is observed that immersive experiences serve as a facilitating resource in the acquisition of abstract or complex concepts because "Among the numerous advantages of this resource, it has the advantage to make abstract or hard-to-access concepts/facts easier to understand by students..." (P_42).

Nevertheless, there are also some concerns and challenges related to cost and resource availability. For example, among the responses, it is mentioned that not all schools can afford necessary resources to work with Immersive Virtual Reality, making some disparities between different schools. (P_28). On the other hand, the potential loss of control during classes concerns teachers, because it could become a distractor during the teaching process: it is a very distracting element, depending on the type of student, you can use it or not, you must be aware of every detail in class... (P_10). Another important factor in the use of Immersive Virtual Reality relates to the shortcomings in Teacher Training: "Limited teacher literacy to deliver classes" (P_57), because "Not all teachers have the skills to use this type of tool." (P_33).

Figure 6
Qualitative Results

Qualitative Results
Source: elaborated by authors. Note: Values expressed in percentage.

DISCUSSION AND CONCLUSIONS

The quantitative results obtained in this research are supported by the qualitative assessments provided by the participants. These qualitative assessments highlight a positive perception of Immersive Virtual Reality in educational contexts, emphasizing its impact on student motivation and engagement. These results are aligned with other research that has also concluded that the use of Augmented Reality (Del Moral et al., 2022; Nikimaleki & Rahimi, 2022; Martínez Pérez & Fernández Robles, 2018) and Immersive Virtual Reality (Álvarez et al., 2023; Radianti et al., 2020; Silva-Díaz et al., 2021) significantly enhances student motivation and engagement. It may also improve the learning process, especially in the field of science. Furthermore, it has been identified that immersive experiences can be beneficial for understanding abstract or complex concepts and suggests that these technologies make these concepts more comprehensible to students (Chang et al., 2019; Cheng & Tsai, 2020; Liu et al., 2020).

Moreover, concerns and challenges related to the use of Virtual Reality, especially Immersive Virtual Reality, in education are also evident. One of the main challenges that has been identified relates to the cost and availability of the resources required to implement technology, which can create inequalities among educational institutions. These concerns have already been observed in previous studies (García-Vandewalle et al., 2022; Silva-Díaz et al., 2021). Additionally, there is some concern about the potential loss of classroom control by teachers during lessons, as Immersive Virtual Reality can become a distracting element in the teaching process if it is not managed properly (Barroso et al., 2019; Nistor et al., 2019). Another relevant aspect to be considered is the lack of teacher training in the use of these tools. This study shows evidence of promoting digital literacy training for teachers to fully leverage the potential of Immersive Virtual Reality in educational settings (Boel et al., 2023; Del Moral et al., 2022; Nistor et al., 2019; Pellas et al., 2019).

Overall, the qualitative assessments complement and support the quantitative findings by providing a more detailed and contextualized perspective of participants' perceptions. These assessments reveal the importance of considering both the benefits and challenges associated with the use of Immersive Virtual Reality in education. They also emphasize the need to address aspects such as accessibility, resource management, classroom control, and teacher training to maximize the benefits of this emerging technology in the teaching and learning processes.

The main findings of this research provide relevant information in the design of strategies that promote the development of technological skills and better integration of technologies in educational contexts in order to foster learning and teaching in the STEM field.

To conclude, there is significant interest in future teachers in the use of various Emerging Technologies, and the high potential attributable to technology, especially in immersive technologies. This demonstrates that their integration into educational environments can enhance motivation, engagement, and content comprehension, and promote more immersive and meaningful learning experiences.

Furthermore, the importance of developing Initial Teacher Training strategies that introduce students to these types of technologies, and also allow them to understand how to integrate them into the classroom is emphasized. It also highlights the need to incorporate these technologies into teacher training programs to improve pedagogical practices.

In future research, the evidence suggests the exploration of innovative approaches for teacher training and addressing the practical implications and advantages of immersive technologies in various educational contexts.

It is important to consider the following limitations when interpreting the results of this research. Firstly, the Seminar (TEC-STEM) on Emerging Technologies in Education has experienced some changes over time, which may have affected the results in different phases of the study. Secondly, the variability in the sample size in the pre and post measurements should be considered. However, it is important to mention that measures were taken to minimize any bias related to sample size variation (application of the Wilcoxon test, which considers differences in sample size when comparing pre and post mean scores).

Acknowledgments

Grants 72210150 and 72210075 awarded by the Chilean Government through “Agencia Nacional de Investigación y Desarrollo (ANID)”. Project TED2021-129474B-I00 funded by MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU / PRTR.

REFERENCES

Aguilera, D., & Perales-Palacios, F. J. (2018). What effects do didactic interventions have on students’ attitudes towards science? A meta-analysis. Research in Science Education, 50, 573-597. https://doi.org/10.1007/s11165-018-9702-2

Álvarez, I. M., Manero, B., Morodo, A., Suñé-Soler, N., & Henao, C. (2023). Realidad Virtual Inmersiva para mejorar la competencia de gestión del clima del aula en secundaria. Educación XX1, 26(1), 249-272. https://doi.org/10.5944/educxx1.33418

Barroso, J., Gutiérrez-Castillo, J. J., Llorente-Cejudo, M. C., & Valencia, R. (2019). Difficulties in the incorporation of Augmented reality in university education: Visions from the experts. Journal of New Approaches in Educational Research, 8(2), 126. https://doi.org/10.7821/naer.2019.7.409

Boel, C., Rotsaert, T., Valcke, M., Rosseel, Y., Struyf, D., & Schellens, T. (2023). Are teachers ready to immerse? Acceptance of mobile immersive virtual reality in secondary education teachers. Research in Learning Technology, 31. https://doi.org/10.25304/rlt.v31.2855

Buss, R. R., Foulger, T. S., Wetzel, K., & Lindsey, L. (2018). Preparing teachers to integrate technology into K–12 instruction II: Examining the effects of technology-infused methods courses and student teaching. Journal of Digital Learning in Teacher Education, 34(3), 134-150. https://doi.org/10.1080/21532974.2018.1437852

Bybee, R. W. (2013). The case of STEM education: challenges and opportunities. National Science Teachers Association.

Cabello, V. M., Martinez, M. L., Armijo, S., & Maldonado, L. (2021). Promoting STEAM learning in the early years: “Pequeños Científicos” Program. LUMAT: International Journal on Math, Science and Technology Education, 9(2). https://doi.org/10.31129/LUMAT.9.2.1401

Cabero-Almenara, J., Romero-Tena, R., Llorente-Cejudo, C., & Palacios-Rodríguez, A. (2021). Academic Performance and Technology Acceptance Model (TAM) Through a Flipped Classroom Experience: Training of Future Teachers of Primary Education. Contemporary Educational Technology, 13(3), ep305. https://doi.org/10.30935/cedtech/10874

Cabero-Almenara, J., Vázquez-Cano, E., Villota-Oyarvide, W. R., & López-Meneses, E. (2021). La innovación en el aula universitaria a través de la realidad aumentada. Análisis desde la perspectiva del estudiantado español y latinoamericano. Revista Electrónica Educare, 25(3), 1-17. https://doi.org/10.15359/ree.25-3.1

Cardona, M. (2002). Introducción a los métodos de investigación en educación. EOS.

Chang, S. C., Hsu, T. C., Kuo, W. C., & Jong, M. S. Y. (2019). Effects of applying a VR-based two-tier test strategy to promote elementary students’ learning performance in a Geology class. British Journal of Educational Technology, 51(1), 148-165. https://doi.org/10.1111/bjet.12790

Cheng, K. H., & Tsai, C. C. (2020). Students’ motivational beliefs and strategies, perceived immersion and attitudes towards science learning with immersive virtual reality: A partial least squares analysis. British Journal of Educational Technology, 51(6) 2139-2158. https://doi.org/10.1111/bjet.12956

Chng, E., Tan, A. L., & Tan, S. C. (2023). Examining the use of emerging technologies in schools: A review of artificial intelligence and immersive technologies in STEM education. Journal for STEM Education Research. https://doi.org/10.1007/s41979-023-00092-y

Christensen, R. (2002). Effects of technology integration education on the attitudes of teachers and students. Journal of Research on Technology in Education, 34(4), 411-433. https://doi.org/10.1080/15391523.2002.10782359

Cviko, A., McKenney, S., & Voogt, J. (2014). Teacher roles in designing technology-rich learning activities for early literacy: A cross-case analysis. Computers & Education, 72, 68-79. https://doi.org/10.1016/j.compedu.2013.10.014

Del Moral Pérez, M. E., Neira Piñeiro, M. R., Castañeda Fernández, J., & López-Bouzas, N. (2022). Competencias docentes implicadas en el diseño de Entornos Literarios Inmersivos: conjugando proyectos STEAM y cultura maker. RIED-Revista Iberoamericana de Educación a Distancia, 26(1), 59-82. https://doi.org/10.5944/ried.26.1.33839

Dubé, A. K., & Wen, R. (2022). Identification and evaluation of technology trends in K-12 education from 2011 to 2021. Education and Information Technologies, 27(1), 1929-1958. https://doi.org/10.1007/s10639-021-10689-8

Ertmer, P. A., Ottenbreit-Leftwich, A. T., Sadik, O., Sendurur, E., & Sendurur, P. (2012). Teacher beliefs and technology integration practices: A critical relationship. Computers & Education, 59(2), 423–435. https://doi.org/10.1016/j.compedu.2012.02.001

Ferrada, C., Carrillo-Rosúa, F. J., Díaz-Levicoy, D., & Silva-Díaz, F. (2020). La robótica desde las áreas STEM en Educación Primaria: una revisión sistemática. Education in the Knowledge Society (EKS), 21(0), 18. https://doi.org/10.14201/eks.22036

Fleer, M. (2013). Affective imagination in science education: determining the emotional nature of scientific and technological learning of young children. Research in Science Education, 43(5), 2085-2106. https://doi.org/10.1007/s11165-012-9344-8

Freeman, A., Adams, S., Cummins, M., Davis, A., & Hall, C. (2017). NMC/CoSN Horizon Report: 2017 K-12 Edition. The New Media Consortium.

García-Vandewalle, J. M., García-Carmona, M., Trujillo Torres, J. M., & Moya-Fernández, P. (2022). The integration of emerging technologies in socioeconomically disadvantaged educational contexts. The view of international experts. Journal of Computer Assisted Learning, 38(4), 1185-1197. https://doi.org/10.1111/jcal.12677

Hernández Sampieri, R., Fernández Collado, C., & Baptista Lucio, M. (2014). Metodología de la investigación (6th ed.). McGraw-Hill Education.

Hod, Y. (2017). Future learning spaces in schools: Concepts and designs from the learning sciences. Journal of Formative Design in Learning, 1(2), 99-109. https://doi.org/10.1007/s41686-017-0008-y

Hung, D., & Khine, M. S. (Eds.). (2006). Engaged learning with emerging technologies. Springer-Verlag. https://doi.org/10.1007/1-4020-3669-8

Liu, R., Wang, L., Lei, J., Wang, Q., & Ren, Y. (2020). Effects of an immersive virtual reality‐based classroom on students’ learning performance in science lessons. British Journal of Educational Technology, 51(6), 2034-2049. https://doi.org/10.1111/bjet.13028

Lui, M., & Slotta, J. D. (2014). Immersive simulations for smart classrooms: exploring evolutionary concepts in secondary science. Technology Pedagogy and Education, 23(1), 57-80. https://doi.org/10.1080/1475939X.2013.838452

Makokha, J. (2017). Emerging technologies and science teaching. In K. S. Taber & B. Akpan (eds.), Science education (pp. 369-383). Sense Publishers. https://doi.org/10.1007/978-94-6300-749-8_27

Makransky, G., Petersen, G. B., & Klingenberg, S. (2020). Can an immersive virtual reality simulation increase students’ interest and career aspirations in science? British Journal of Educational Technology, 51(6), 2079-2097. https://10.1111/bjet.12954

Martín-Páez, T., Aguilera, D., Perales-Palacios, F. J., & Vílchez-González, J. M. (2019). What are we talking about when we talk about STEM education? A review of literature. Science Education, 103(4), 799–822. https://doi.org/10.1002/sce.21522

Martínez Pérez, S., & Fernández Robles, B. (2018). Objetos de Realidad Aumentada: percepciones del alumnado de Pedagogía. Pixel bit, 53, 207-220. https://doi.org/10.12795/pixelbit.2018.i53.14

Nikimaleki, M., & Rahimi, M. (2022). Effects of a collaborative AR‐enhanced learning environment on learning gains and technology implementation beliefs: Evidence from a graduate teacher training course. Journal of Computer Assisted Learning, 38(3), 758-769. https://doi.org/10.1111/jcal.12646

Nistor, N., Stanciu, D., Lerche, T., & Kiel, E. (2019). “I am fine with any technology, as long as it doesn’t make trouble, so that I can concentrate on my study”: A case study of university students’ attitude strength related to educational technology acceptance. British Journal of Educational Technology: Journal of the Council for Educational Technology, 50(5), 2557-2571. https://doi.org/10.1111/bjet.12832

Pellas, N., Fotaris, P., Kazanidis, I., & Wells, D. (2019). Augmenting the learning experience in primary and secondary school education: a systematic review of recent trends in augmented reality game-based learning. Virtual Reality, 23(4), 329-346. https://doi.org/10.1007/s10055-018-0347-2

Radianti, J., Majchrzak, T. A., Fromm, J., & Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education, 147(103778). https://doi.org/10.1016/j.compedu.2019.103778

Sanders, M. (2009). STEM, STEM education, STEMmania. The Technology Teacher, 68(4), 20-27.

Sánchez-Prieto, J. C., Huang, F., Olmos-Migueláñez, S., García-Peñalvo, F. J., & Teo, T. (2019). Exploring the unknown: The effect of resistance to change and attachment on mobile adoption among secondary pre‐service teachers. British Journal of Educational Technology: Journal of the Council for Educational Technology, 50(5), 2433-2449. https://doi.org/10.1111/bjet.12822

Silva-Díaz, F., Carrillo-Rosúa, J., & Fernández-Plaza, J. A. (2021). Uso de tecnologías inmersivas y su impacto en las actitudes científico-matemáticas del estudiantado de Educación Secundaria Obligatoria en un contexto en riesgo de exclusión social. Educar, 57(1), 119-138. https://doi.org/10.5565/rev/educar.1136

Silva-Díaz, F., Fernández-Ferrer, G., Vázquez-Vílchez, M., Ferrada, C., Narváez, R., & Carrillo-Rosúa, J. (2022). Tecnologías emergentes en la educación STEM. Análisis bibliométrico de publicaciones en Scopus y WoS (2010-2020). Bordón. Revista de Pedagogía, 74(4), 25-44. https://doi.org/10.13042/Bordon.2022.94198

Thibaut, L., Ceuppens, S., De Loof, H., De Meester, J., Goovaerts, L., Struyf, A., Boeve-de Pauw, J., Dehaene, W., Deprez, J., De Cock, M., Hellinckx, L., Knipprath, H., Langie, G., Struyven, K., Van de Velde, D., Van Petegem, P., & Depaepe, F. (2018). Integrated STEM education: A systematic review of instructional practices in secondary education. European Journal of STEM Education, 3(1). https://doi.org/10.20897/ejsteme/85525

Toma, R. B., & Greca, I. M. (2018). The effect of integrative STEM instruction on elementary students’ attitudes toward science. Eurasia Journal of Mathematics, Science and Technology Education, 14(4), 1383-1395. https://doi.org/10.29333/ejmste/83676

Toma, R. B., & Meneses-Villagrá, J. A. (2019). Preferencia por contenidos científicos de física o de biología en educación primaria: un análisis clúster. Revista Eureka sobre Enseñanza y Divulgación de las Ciencias, 16(1), 1104. https://doi.org/10.25267/Rev_Eureka_ensen_divulg_cienc.2019.v16.i1.1104

Xia, L., & Zhong, B. (2018). A systematic review on teaching and learning robotics content knowledge in K-12. Computers & Education, 127, 267-282. https://doi.org/10.1016/j.compedu.2018.09.007

Zollman, A. (2012). Learning for STEM literacy: STEM literacy for learning. School Science and Mathematics, 112(1), 12-19. https://doi.org/10.1111/j.1949-8594.2012.00101.x

Notes


Date of reception: 01 June 2023
Date of acceptance: 26 August 2023
Date of publication in OnlineFirst: 02 October 2023
Date of publication: 01 January 2024