INTRODUCTION

Computational thinking (CT) is increasingly present in various educational systems. The more intuitive technological developments and the supposed benefits of its development make it an essential competence (González et al., 2018). Wing (2006) defined CT as a thinking process based on computational concepts that enables problem solving and system design. Following Wing’s publication, research in this area experienced a growing increase. However, studies addressing teacher training and pedagogical approaches to CT teaching and learning remain scarce. Most focus on measuring CT (Haseski & Ilic, 2019) and reflect the lack of consensus on its definition and components (Bocconi et al., 2016; Hsu et al., 2019; Moreno-León et al., 2019; Román-González et al., 2017; Segredo et al., 2017; Shute et al., 2017).

Aware of the need for agreement on these aspects, Angeli et al. (2016, p.49) attempted to unify existing definitions. The authors define CT as "a thinking process that uses the elements of abstraction, generalisation, decomposition, algorithmic thinking and debugging (error detection and correction)". There is also a lack of consensus on other issues, such as its scope or nature (González et al., 2018; Rich & Langton, 2016). This continues to highlight the need to further delimit the term (Bocconi et al., 2016) in order for it to be fully and effectively included in education.

In most countries, the integration of CT takes place mainly at the secondary education stage (Haseski & Ilic, 2019). However, it is necessary to include it from elementary education levels in the same way as other basic skills such as numeracy or reading (Zapata-Ros, 2015). CT can enable the development of skills such as creativity, problem solving or collaborative skills (Arranz & Pérez, 2017). Including it in primary school would also help to alleviate gender differences in CT development (Ketelhut et al., 2019; Shute et al., 2017). However, for this inclusion to be truly effective, a number of methodological and pedagogical criteria must be established. There is a clear need for further studies in this direction, as they are still scarce ( Haseski & Ilic, 2019).

Another aspect that also requires further study is the evaluation of CT (Bocconi et al., 2016; Cutumisu et al., 2019, Lockwood & Mooney, 2017; Román-González et al., 2017). Existing tools and methods cover only certain aspects, focusing more on computational concepts than on practices or perspectives. The existence of multiple definitions of CT has generated a wide variety of assessment methods that lack sufficient validation (Cutumisu et al., 2019; Shute et al., 2017).

Teachers play a key role in the inclusion of CT in schools (Yadav et al., 2017). Teacher qualifications are considered a major factor for educational quality and an essential driver for effectively addressing educational change. The introduction of CT to education policy is creating a strong demand for teacher training, as most teachers do not learn about CT in their initial training (Bocconi et al., 2016; Bustillo, 2015; Ling et al., 2018; Yadav et al., 2017). Given the shortage of teachers qualified to teach this type of thinking and effectively use tools to foster its development (Ling et al., 2018), more studies focusing on this group (Haseski & Ilic, 2019) are needed to design evidence-based training plans.

Several researchers have addressed some of these issues in systematic reviews or literature reviews conducted more frequently since 2014. Works such as Kalelioğlu et al. (2016) and Cometa et al. (2021) provide an overview of the status of CT. Shute et al. (2017) analyse different aspects related to this thinking (characteristics and components, interventions to develop it, evaluation and existing theoretical frameworks and models). Others focus on specific dimensions. Pollak and Ebner (2019) investigate the integration of CT, and Palts and Pedaste (2020) discuss the dimensions of CT thinking skills and how they can be combined in a new model to develop CT thinking. Cutumisu et al. (2019), Alves et al. (2019) and Tang et al. (2020) discuss, from different perspectives, the assessment of CT. Ioannou and Makridou (2018), Uslu et al. (2022) and Xia and Zhong (2018) focus on robotics, while Lye and Koh (2014), Popat and Starkey (2018) and Zhang and Nouri (2019) analyse different aspects related to coding. Other studies, such as García-Tudela and Marín-Marín (2023), are more specific and analyse the uses and achievements of Arduino. Barcelos et al. (2019) and Chan et al. (2023) address the development of mathematical learning through CT activities. Menon et al. (2019) investigate the use of escape board games to develop and assess CT.

Although the authors of the aforementioned systematic reviews address some of the issues surrounding CT, it remains unclear whether there are methods, specific strategies or tools that can be used to successfully incorporate it from the primary education stage onwards. The search for these answers, which is the subject of this new systematic review, contributes to the subsequent identification of the keys to defining what initial teacher education in universities should be like. We believe that school practice influences and provides valuable information for the design of curricula in faculties of education.

In this context, this systematic review seeks answers to the following research questions:

What are the main learning strategies, inclusion principles and tools used in the implementation of CT at the primary education stage (6-12 years old)?

What are the main characteristics of the design of initial and in-service training experiences for primary school teachers in CT?

METHOD

In this study, evidence is gathered on the current situation regarding teacher training and the implementation of CT in primary education. The general objective is to draw conclusions about the teaching-learning process of CT in primary education that can be considered in university teacher training.

The research design follows the process established by Kitchenham and Charters (2007) for conducting systematic reviews and the recommendations offered by Sánchez-Meca (2022). The guidance and checklist of the PRISMA 2020 Declaration (Page et al., 2021) are also accounted for. Annex I shows which suggestions from this checklist have been met, which have not been met, and which are not appropriate to check, as this is not a meta-analysis.

Selection criteria

The selection of studies was based on the criteria set out in Table 1.

In 2006, Wing published his article titled Computational Thinking, which, together with international education policies and the impetus of the technology industry, led to an increase in the popularity of the term and interest in its study and inclusion in the classroom. This milestone justifies the criterion that eligible studies were published between 2006 and 14 March 2023, the day in which the search was conducted.

Search strategies

The Scopus and Web of Science (WOS) databases were used to search for articles, following the unregistered protocol set out in Table 2.

Procedure for the selection of studies

After an initial search in Scopus and WOS, 231 and 197 articles were obtained, respectively. A total of 168 duplicate articles were eliminated. Subsequently, the titles were read, and 63 articles were eliminated because they were not clearly related to the research objectives. Next, the abstracts were read, and 92 articles were eliminated. After a thorough reading, the selection was reduced to 47 studies. Finally, the quality of these articles was assessed (the procedure is detailed in the following section), and 24 articles were selected, as shown in Figure 1. This whole process was carried out by the researcher who is the first author of this article.

Assessment of the quality of studies

The Critical Appraisal Skill Programme (CASP) checklist¹ was used to assess the quality of the articles. This tool, developed by the Oxford Centre for Triple Value Healthcare, allows the critical appraisal of the reliability and relevance of the results obtained in the different studies, regardless of the area of knowledge from which they originate. It provides ten questions that are accompanied by a series of indications for answering them correctly. With the application of the CASP checklist, the aim is to eliminate the risk of bias to guarantee the credibility and generalizability of the results obtained.

To assess the different types of studies, an adaptation of the CASP checklist including nine questions was created. The quality of the studies was assessed independently by two investigators, as shown in Annex II. Papers were labelled high quality (≥70%), moderate quality (69-40%) or low quality (<40%). Only articles of high and moderate quality (n=24) were included. Excluded articles are shown in Annex III.

RESULTS

Following the procedure developed in the previous section, a total of 24 articles were selected. Analysing their general characteristics revealed that most of the articles were published from 2019 onwards, with a concentration in 2021. Most of the studies took place in China, followed by the United States and Spain. The continents of Asia and Europe accounted for most of the selected articles. Analysing the general contributions of the research, 58.3% were based on the description of CT experiences with students or teachers and future primary school teachers, and 37.5% were based on the design and validation of CT models, programmes, resources or evaluation instruments. In terms of research design, most of the studies used quantitative or mixed methods. Experiments with an experimental and control group and the use of pretest and posttest predominated.

In relation to the research questions and the objectives of the review, most of the articles were related to the implementation and evaluation of experiences with CT among primary school students. Only 6 of the 24 articles dealt with the training of in-service or future teachers. The most relevant results are developed below:

Implementing computational thinking in primary education

Most studies focused on students in the third to sixth grades of primary school (Basu et al., 2021; Chevalier et al., 2020; Chiazzese et al., 2019; El-Hamamsy et al., 2022; Freina et al, 2019; Hooshyar et al., 2021; Huang et al.,2023; Jiang & Li, 2021; Liu et al., 2023; Noh & Lee, 2019; Sáez & Cózar, 2017; Sengupta et al., 2013; Shen et al., 2020; Tran, 2019; Wei et al., 2021). Only four studies (Caballero-González & García-Valcárcel, 2020; Del Olmo et al., 2020; El-Hamamsy et al., 2022; Yadav & Chakraborty, 2023) addressed initial levels.

Regarding the tools used, most studies employed robotics (Caballero-González & García-Valcárcel, 2020; Chevalier et al., 2020; Chiazzese et al., 2019; Liu et al, 2023; Noh & Lee, 2019; Shen et al., 2020) and/or computer programming (Basu et al., 2021; Freina et al., 2019; Jiang & Li, 2021; Sáez & Cózar, 2017; Sengupta et al., 2013; Tran, 2019; Wei et al., 2021). Most did so through visual programming environments, such as Scratch (Basu et al., 2021; Chevalier et al., 2020; Freina et al., 2019; Jiang & Li, 2021; Sáez & Cózar, 2017; Sengupta et al., 2013; Wei et al., 2021), App Inventor (Basu et al., 2021) or Blockly (Tran, 2019).

Only two papers (Basu et al., 2021; Del Olmo et al., 2020) addressed CT without using technological devices. Other studies made use of computer games (Hooshyar et al., 2021) or board games enriched with augmented reality (Huang et al., 2023). Finally, apps were also used, such as Lightbot, Code Hour, Q Space App, Preschool and kindergarten app and Luca's educational patterns game app (Yadav & Chakraborty, 2023).

With regard to the learning strategies followed to develop CT in primary school, learning by doing stands out (Chiazzese et al., 2019; Freina et al., 2019; Hooshyar et al., 2021; Jiang & Li, 2021; Liu et al., 2023; Sáez & Cózar, 2017; Shen et al, 2020; Yadav & Chakraborty, 2023), followed by collaborative work (Caballero-González & García-Valcárcel, 2020; Chevalier et al., 2020; Chiazzese et al., 2019; Freina et al., 2019; Jaipal-Jamani & Angeli, 2016; Jiang & Li, 2021; Liu et al, 2023; Noh & Lee, 2019; Tran, 2019; Wei et al., 2021), scaffolding (Chevalier et al., 2020; Freina et al., 2019; Jaipal-Jamani & Angeli, 2016; Sengupta et al., 2013; Yadav & Chakraborty, 2023), and solving problems or challenges (Basu et al., 2021; Caballero-González & García-Valcárcel, 2020; Chevalier et al., 2020; Chiazzese et al., 2019; Freina et al., 2019; Jiang & Li, 2021; Liu et al., 2023; Shen et al., 2020; Yadav & Chakraborty, 2023).

Other studies revealed some principles to consider for CT inclusion to be truly effective: taking students' prior skills into account (Chiazzese et al., 2019; Huang et al.,2023; Liu et al., 2023; Noh & Lee, 2019; Sengupta et al, 2013); using varied environments, tools and activities to respond to the diversity of learners (Sengupta et al., 2013; Tran, 2019); and developing CT from the lowest levels (Caballero-González & García-Valcárcel, 2020; El-Hamamsy et al., 2022; Del Olmo et al., 2020; Tran, 2019; Yadav & Chakraborty, 2023). Given the near nonexistence of methodological and pedagogical principles for teaching CT in primary school, some authors designed their own models and approaches (Chevalier et al., 2020; Liu et al., 2023; Sengupta et al., 2013).

Several studies presented examples of the inclusion of CT in curricular areas, especially in STEM areas such as computer science, science and mathematics (Chiazzese et al., 2019; Sengupta et al., 2013; Tran, 2019; Yadav & Chakraborty, 2023) but also others discussed this in areas such as art education (Sáez & Cózar, 2017). The remaining studies created unique programmes without integrating them into specific curricular areas.

The most commonly used assessment instrument was the selection and adaptation of Bebras tests (Chiazzese et al., 2019; Del Olmo et al., 2020; Huang et al., 2023; Noh & Lee, 2019; Yadav & Chakraborty, 2023). Some authors (Hooshyar et al., 2021; Jiang & Li, 2021; Liu et al., 2023) used or adapted validated instruments, such as the CT test (CCT) by Román-González et al. (2015), the CT scale (CTS) by Korkmaz et al. (2017) or Dr. Scratch by Moreno-León et al. (2015). Other authors created their own assessment instruments developed specifically for the experience conducted (Basu et al., 2021; El-Hamamsy et al., 2022; Shen et al., 2020; Tran, 2019). Authors such as El-Hamamsy et al. (2022) highlighted the importance of combining several CT tests to adequately assess students' computational competences.

Teacher training in computational thinking

Effective inclusion of CT in primary classrooms requires teachers to possess a range of pedagogical skills and knowledge. However, the reality is that most teachers have no prior CT training or have misconceptions about CT (Chalmers, 2018; Freina et al., 2019; Jaipal-Jamani & Angeli, 2016). However, we found studies showing that teachers experience an increase in their computer skills and confidence in including CT in the classroom after participating in training experiences (Chalmers, 2018; Jaipal-Jamani & Angeli, 2016; Kong & Lai, 2022; Molina-Ayuso et al., 2022; Rich et al., 2021).

Most of the teacher education experiences reviewed were aimed at in-service teachers (Chalmers, 2018; Kong et al., 2023; Kong & Lai, 2022; Rich et al., 2021). Only two studies (Jaipal-Jamani & Angeli, 2016; Molina-Ayuso et al., 2022) targeted prospective primary school teachers. The duration of these training experiences was quite variable, with some experiences lasting less than three months (Chalmers, 2018; Jaipal-Jamani & Angeli, 2016) and others lasting more than eight months (Kong et al., 2023; Kong & Lai, 2022; Rich et al., 2021).

Regarding the content of these learning experiences, some papers addressed computer programming (Kong et al., 2023; Kong & Lai, 2022; Molina-Ayuso et al., 2022; Rich et al., 2021) or robotics (Chalmers, 2018; Jaipal-Jamani & Angeli, 2016). Only the study by Kong and Lai (2022) considered the use of disconnected activities. Among the works which addressed computer programming, those describing the use of Scratch or Scratch Jr (Kong et al., 2023; Kong & Lai, 2022; Molina-Ayuso et al., 2022; Rich et al., 2021) stand out, although other resources, such as App Inventor (Kong et al., 2023; Kong & Lai, 2022) are also used. Robotics is addressed with Lego WeDo kits (Chalmers, 2018; Jaipal-Jamani & Angeli, 2016).

Analysing the structure of the courses reveals that most of them opt for an initial instruction phase and a practice or knowledge implementation phase (Jaipal-Jamani & Angeli, 2016; Kong et al., 2023; Kong & Lai, 2022; Molina-Ayuso et al., 2022; Rich et al., 2021). The most commonly employed strategies are collaborative work (Jaipal-Jamani & Angeli, 2016; Kong et al., 2023), modelling (Kong & Lai, 2022; Rich et al., 2021) and scaffolding (Jaipal-Jamani & Angeli, 2016; Kong et al., 2023).

Finally, the evaluation of training experiences was very diverse and was carried out using a variety of tests created specifically for the experience or adapted from existing ones. All studies assessed both teachers' knowledge acquisition and their reflections on the experience. Studies such as Chalmers (2018) and Kong and Lai (2022) are relevant because they provided information on the challenges teachers face.

Tables 3 and 4 show a synthesis of the above results.

DISCUSSION

In this section, we present a general interpretation of the results in the context of other studies and discuss their implications for educational practice and future research. In addition, we show in the final part of this section the most substantial limitations we identified from this systematic review.

First, we contrasted our results with those obtained in other research on the implementation of CT in primary education (first research question).

Although education systems in many countries have modified their curricula to respond to the need for the efficient inclusion of CT in primary classrooms, we have found that there are very few studies that focus on the first two years of primary school, which undermines the desirability of starting CT development early. This also makes it difficult to recommend a gradual shift from offline to technology-based activities (Brackmann et al., 2017; Serrano & Ortuño, 2021).

Visual block programming and robotics were the most commonly used strategies or tools in the reviewed studies, coinciding with the conclusions obtained in two meta-analyses (Merino-Armero et al., 2021; Sun et al., 2021), which indicated that robot programming is the most efficient strategy for CT development in primary education.

The use of disconnected activities was fairly limited in the selected studies. This contrasts with the advantages, especially in primary school (Serrano & Ortuño, 2021), that this strategy has according to several previous studies (Huang & Looi, 2020; Weigend et al., 2019). In one meta-analysis (Li et al., 2022), the authors also concluded that both unplugged activities and scheduling exercises are useful for CT development among students. According to this study, the effects of programming are somewhat better than those provided by disconnected activities and are enhanced when working in an interdisciplinary way and not only within specific subjects, which is far from the results of our research.

In relation to the teaching strategies employed, we concluded that there are a number of strategies that are suitable for use, including instructional and learning-by-doing strategies, scaffolding, collaborative work and problem or challenge solving, thus largely agreeing with previous work (Park & Park, 2018; Kale et al., 2018; Voogt et al., 2015). However, game-based learning was reported in some previous studies (Kaleli̇oğlu et al., 2016) as one of the most commonly used strategies in CT development experiences, a conclusion that does not coincide with the results obtained in our review.

Some authors analysed how to assess CT development, showing that there is a great gap in terms of valid assessment tools and strategies (Alves et al., 2019). Although there are instruments for assessing CT development in primary education (Bebras, CCT Test, CTS Scale, Opinion polls, Dr. Scratch, etc.), we find that they are not useful for verifying the real and complete development of CT, as they focus more on programming skills than on analysing the development of CT components. In the absence of effective approaches to assess CT in primary school, we propose the combination of multiple assessments to cover all dimensions of CT. This issue requires further research, as already noted in previous studies (Cutumisu et al., 2019; Lockwood & Mooney, 2017; Román-González et al., 2017).

Second, we discuss the characteristics of CT teacher education at the primary school level (second research question).

According to the results obtained, we note the scarcity of studies that rigorously evaluate training experiences, especially in initial teacher training, coinciding with the results of other research (Bocconi et al., 2016; Bustillo, 2015; Ling et al., 2018; Yadav et al., 2017). This is most likely because there is still no proliferation of teacher training plans for CTdevelopment, which is still a pending goal, as pointed out by Haseski and Ilic (2019). Educational institutions should take the lead in teacher training, as they are the best places for teachers to reflect on their previous beliefs about CT, to learn about key concepts and, most importantly, to adopt the most appropriate pedagogical approaches (González et al., 2018; Yadav et al., 2017).

Introducing CT concepts in the initial training of teachers is a key strategy for achieving this goal since it is at that time that they are generally most willing to understand the relevance of CT within their own discipline (Butler & Leahy, 2021; Yadav et al., 2014). In any case, beyond specific training actions, the implementation and evaluation—in different contexts—of training proposals is needed, and fortunately, such efforts are beginning to emerge and can be consulted in different works (Esteve-Mon et al., 2019; Kotsopoulos et al., 2017; Serrano & Ortuño, 2021; Tsai et al., 2021; Voon et al., 2023).

In addition to the need to design training models that operationalize CT development with specific strategies, importantly, there are generic pedagogical models that can be the basis for designing specific models. Some authors (Kale et al., 2018; Yadav et al., 2017) consider the TPACK (technology, pedagogy, and content knowledge) model, promoted by Mishra and Koehler (2006), to be useful for teaching CT in initial teacher education. This model includes preservice teachers learning about the effective integration of technology within the context of subject matter and pedagogy; similarly, teachers need to develop CT knowledge within the context of their content knowledge and pedagogical knowledge.

The study by Kong et al. (2020) presented a (primary) teacher development programme to develop CT competences in relation to programming and the TPACK model. The researchers identified concerns among teachers about programming due to the lack of efficient and pedagogically rich training support. The authors proposed a pedagogy for playing, thinking and coding for the development of computer skills with programming in primary schools.

Despite the emphasis on CT as a mental tool that extends beyond computer-based environments, most current teacher professional development efforts have focused on exposing teachers to programming environments, consistent with the findings of our review. Thus, teachers should be engaged to think about how CT can be integrated into authentic learning situations in other content areas. The training programme developed by Yadav et al. (2018) highlights this idea.

The pedagogical knowledge training of primary school teachers is, in general, solid. However, this needs to be complemented by CT-specific pedagogical practices and content, such as modelling a problem, thinking about or solving a problem iteratively and incrementally, or explaining a solution to a problem in a series of steps (Carlborg et al., 2019).

Furthermore, we found that in-service teachers carry out training that varies in duration and is very focused on technical aspects such as computer programming and robotics, leaving in the background the use of unplugged activities and issues related to didactic strategies, precisely those aspects that have a high impact on learning in promoting CT among future students, as mentioned above.

In summary, we conclude that – most likely – many teachers are trying to integrate CT into their classrooms without having the necessary competences, especially those that should be developed during initial teacher training. Such training should be based on the principles of educational technology, involve teachers in the task of determining how to integrate CT into learning situations in an interdisciplinary way, be complemented by pedagogical practices and CT content, go beyond teaching programming, and incorporate offline activities as a preliminary step in robot programming. The effectiveness of these training programmes requires improved approaches to CT development assessment beforehand.

Third, we refer to the main limitations of the research. The remaining limitations, which we consider minor, can be found in the PRISMA 2020 checklist (Annex I).

This systematic review analyses the studies collected by Scopus and WOS. With this inclusion criterion, we deliberately discarded other types of publications that could be equally valuable. This decision was made because previous systematic reviews did account for studies published in journals indexed in databases such as ERIH Plus. It would be valuable to consider unpublished "grey literature" on the topic to counteract the problem of publication bias. However, we argue that this limitation does not detract from the validity of the conclusions obtained. Selecting studies published in high-impact journals is a useful, necessary and accepted filter among the scientific community.

Although in the first task of article selection (reading article titles) we discarded those studies that were clearly not related to the research, we consider that in order to avoid bias and errors, it would be more convenient to analyse the abstracts as well. Another aspect to improve in this part in future studies is that the selection of studies should be done by more than one reviewer and in an independent manner. This recommendation was partially followed during the assessment of the quality of the papers with the CASP checklist, in which the two researchers who signed this article participated. In relation to this instrument, we consider that its utility is questionable for two reasons: because it does not distinguish studies according to the type of research design of each study and because of its limited application in studies in the social sciences. We recommend exploring and selecting other checklists or scales available on the Equator Network website².

REFERENCES

Alves, N. D. C., Von Wangenheim, C. G., & Hauck, J. C. (2019). Approaches to Assess Computational Thinking Competences Based on Code Analysis in K-12 Education: A Systematic Mapping Study. Informatics in Education, 18(1), 17-39. https://doi.org/10.15388/infedu.2019.02

Angeli, C., Voogt, J., Fluck, A., Webb, M., Cox, M., Malyn-Smith, J., & Zagami, J. (2016). A K-6 Computational Thinking Curriculum Framework: Implications for Teacher Knowledge. Educational Technology and Society (EtandS), 19(3), 47-57. https://www.j-ets.net/collection/published-issues/19_3

Arranz, H., & Pérez, A. (2017). Evaluación del Pensamiento Computacional en Educación. Revista Interuniversitaria de Investigación en Tecnología Educativa (RIITE), (3), 25-39. https://doi.org/10.6018/riite/2017/267411

Barcelos, T. S., Munoz, R., Villarroel, R., Merino, E., & Silveira, I. F. (2018). Mathematics Learning through Computational Thinking Activities: A Systematic Literature Review. Journal of Universal Computer Science, 24(7), 815-845. https://doi.org/10.3217/jucs-024-07-0815

Basu, S., Rutstein, D. W., Xu, Y., Wang, H., & Shear, L. (2021). A principled approach to designing computational thinking concepts and practices assessments for upper elementary grades. Computer Science Education, 31(2), 169-198. https://doi.org/10.1080/08993408.2020.1866939

Bocconi, S., Chioccariello, A., Dettori, G., Ferrari, A., Engelhardt, K., Kampylis, P., & Punie, Y. (2016). Exploring the field of computational thinking as a 21st century skill. EDULEARN 16, Barcelona, Spain. https://doi.org/10.21125/edulearn.2016.2136

Brackmann, C. P., Román-González, M., Robles, G., Moreno-León, J., Casali, A., & Barone, D. (2017). Development of Computational Thinking Skills through Unplugged Activities in Primary School [Acta]. 12th Workshop on Primary and Secondary Computing Education (WiPSCE), Nijmegen, Países Bajos. https://doi.org/10.1145/3137065.3137069

Bustillo, J. (2015). Formación del profesorado con Scratch: análisis de la escasa incidencia en el aula. Opción, 31(1), 164-182. http://www.redalyc.org/articulo.oa?id=31043005010

Butler, D., & Leahy, M. (2021). Developing preservice teachers’ understanding of computational thinking: A constructionist approach. British Journal of Educational Technology, 52(3), 1060-1077. https://doi.org/10.1111/bjet.13090

Caballero-González, Y., & García-Valcárcel, A. (2020). ¿Aprender con robótica en Educación Primaria? Un medio de estimular el pensamiento computacional. Education in the Knowledge Society, 21, 10. https://doi.org/10.14201/eks.21443

Carlborg, N., Tyrén, M., Heath, C., & Eriksson, E. (2019). The scope of autonomy when teaching computational thinking in primary school. International Journal of Child-Computer Interaction, 21, 130-139. https://doi.org/10.1016/j.ijcci.2019.06.005

Chalmers, C. (2018). Robotics and computational thinking in primary school. International Journal of Child-Computer Interaction, 17, 93-100. https://doi.org/10.1016/j.ijcci.2018.06.005

Chan, S.-W., Looi, C.-K., Ho, W. K., & Kim, M. S. (2023). Tools and Approaches for Integrating Computational Thinking and Mathematics: A Scoping Review of Current Empirical Studies. Journal of Educational Computing Research, 60(8), 2036-2080. https://doi.org/10.1177/07356331221098793

Chevalier, M., Giang, C., Piatti, A., & Mondada, F. (2020). Fostering computational thinking through educational robotics: a model for creative computational problem solving. International Journal of STEM Education, 7, 39. https://doi.org/10.1186/s40594-020-00238-z

Chiazzese, G., Arrigo, M., Chifari, A., Lonati, V., & Tosto, C. (2019). Educational Robotics in Primary School: Measuring the Development of computational Thinking Skills with the Bebras Tasks. Informatics, 6(4), 6040043. https://doi.org/10.3390/informatics6040043

Cometa, V., Parque, S., & Wiebe, E. (2021). The code-centric nature of computational thinking education: a review of trends and issues in computational thinking education research. Sage, 11(2). https://doi.org/10.1177/21582440211016418

Cutumisu, M., Adams, C., & Lu, C. (2019). A Scoping Review of Empirical Research on Recent Computational Thinking Assessments. Journal of Science Education and Technology, 28(1), 651-676. https://doi.org/10.1007/s10956-019-09799-3

Del Olmo, J. del, Cózar-Gutiérrez, R., & González-Calero, J. A. (2020). Computational thinking through unplugged activities in early years of Primary Education. Computers & Education, 150(1), 103832. https://doi.org/10.1016/j.compedu.2020.103832

El-Hamamsy, L., Zapata-Cáceres, M., Martín-Barroso, E., Mondada, F., Dehler-Zufferey, J., & Bruno, B. (2022). The Competent Computational Thinking Test: Development and Validation of an Unplugged Computational Thinking Test for Upper Primary School. Sage, 60(7), 1818-1866. https://doi.org/10.1177/07356331221081753

Esteve-Mon, F., Adell-Segura, J., Llopis, M. A., Valdeoliva, G., & Pacheco, J. (2019). The Development of Computational Thinking in Student Teachers through an Intervention with Educational Robotics. Journal of Information Technology Education: Innovations in Practice, 18, 139-152. https://doi.org/10.28945/4442

Freina, L., Bottino, R., & Ferlino, L. (2019). Fostering Computational Thinking skills in the Last Years of Primary School. International Journal of Serious Games, 6(3), 101-115. https://doi.org/10.17083/ijsg.v6i3.304

García-Tudela, P. A., & Marín-Marín, J. A. (2023). Use of Arduino in Primary Education: a systematic review. Education Sciences, 13(2), 134. https://doi.org/10.3390/educsci13020134

González, J., Estebanell, M., & Peracaula, M. (2018). ¿Robots o programación? El concepto de Pensamiento Computacional y los futuros maestros. Education in the Knowledge Society (EKS), 19(2), 29-45. https://doi.org/10.14201/eks20181922945

Haseski, H. I. e Ilic, U. (2019). An Investigation of the Data Collection Instruments Development to Measure Computational Thinking. Informatics in Education, 18(2), 297-319. https://doi.org/10.15388/infedu.2019.14

Hooshyar, D., Malva, L., Yang, Y., Pedaste, M., Wang, M., & Lim, H. (2021). An adaptive educational computer game: Effects on students' knowledge and learning attitude in computational thinking. Computers in Human Behavior, 114, 106575. https://doi.org/10.1016/j.chb.2020.106575

Hsu, Y.-C., Irie, N. R., & Ching, Y.-H. (2019). Computational Thinking Educational Policy Initiatives (CTEPI) Across the Globe. TechTrends, 63(2), 260-270. https://doi.org/10.1007/s11528-019-00384-4

Huang, S.-Y., Tarng, W., & Ou, K.-L. (2023). Effectiveness of AR Board Game on Computational Thinking and Programming Skills for Elementary School Students. Systems, 11(1), 25. https://doi.org/10.3390/systems11010025

Huang, W., & Looi, C.-K. (2020). A critical review of literature on "unplugged" pedagogies in K-12 computer science and computational thinking education. Computer Science Education, 31(1), 83-111. https://doi.org/10.1080/08993408.2020.1789411

Ioannou, A., & Makridou, E. (2018). Exploring the potentials of educational robotics in the development of computational thinking: A summary of current research and practical proposal for future work. Education and Information Technologies, 23(1), 2531-2544. https://doi.org/10.1007/s10639-018-9729-z

Jaipal-Jamani, K., & Angeli, C. (2016). Effect of Robotics on Elementary Preservice Teachers´ Self-Efficacy, Science Learning and Computational Thinking. Journal of Science Education and Technology, 26(2), 175-192. https://doi.org/10.1007/s10956-016-9663-z

Jiang, B., & Li, Z. (2021). Effect of Scratch on computational thinking skills of Chinese primary school students. Journal of Computers in Education, 8(4), 505-525. https://doi.org/10.1007/s40692-021-00190-z

Kale, U., Akcaoglu, M., Cullen, T., Goh, D., Devine, L., Calvert, N., & Grise, K. (2018). Computational What? Relating Computational Thinking to Teaching. TechTrends, 62(6), 574-584. https://doi.org/10.1007/s11528-018-0290-9

Kalelioğlu, F., Gülbahar, Y., & Kukul, V. (2016). A Framework for Computational Thinking Based on a Systematic Research Review. Baltic Journal Modern Computing, 4(3), 583-596. https://www.bjmc.lu.lv/fileadmin/user_upload/lu_portal/projekti/bjmc/Contents/4_3_15_Kalelioglu.pdf

Ketelhut, D.-J., Mills, K., Hestness, E., Cabrera, L., Plane, J., & McGinnis, J.-R. (2019). Teacher Change Following a Professional Development Experience in Integrating Computational Thinking into Elementary Science. Journal of Science Education and Technology, 29(1), 174-188. https://doi.org/10.1007/s10956-019-09798-4

Kitchenham, B., & Charters, S. (2007). Guidelines for performing Systematic Literature Reviews in Software Engineering. Version 2.3. In EBSE Technical Report EBSE-2007-01. https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.117.471

Kong, S.-C., & Lai, M. (2022). A proposed computational thinking teacher development framework for K-12 guided by the TPACK model. Journal of Computers in Education, 9, 379-402. https://doi.org/10.1007/s40692-021-00207-7

Kong, S.-C., Lai, M., & Li, Y. (2023). Scaling up a teacher development programme for sustainable computational thinking education: TPACK surveys, concept tests and primary school visits. Computers & Education, 194, 104707. https://doi.org/10.1016/j.compedu.2022.104707

Kong, S.-C., Lai, M., & Sun, D. (2020). Teacher development in computational thinking: Design and learning outcomes of programming concepts, practices and pedagogy. Computers & Education, 151, 103872. https://doi.org/10.1016/j.compedu.2020.103872

Korkmaz, Ö., Çakir, R., & Özden, M. Y. (2017). A validity and reliability study of the Computational Thinking Scales (CTS). Computers in Human Behavior, 72, 558-569. https://doi.org/10.1016/j.chb.2017.01.005

Kotsopoulos, D., Floyd, L, Khan, S., Namukasa, I., Somanath, S., Weber, J., & Chris, Y. (2017). A Pedagogical Framework for Computational Thinking. Digital Experiences in Mathematics Education, 3(2), 154-171. https://doi.org/10.1007/s40751-017-0031-2

Li, F., Wang, X., He, X., Cheng, L., & Wang, Y. (2022). The effectiveness of unplugged activities and programming exercises in computational thinking education: A Meta-analysis. Education and Information Technologies, 27, 7993-8013. https://doi.org/10.1007/s10639-022-10915-x

Ling, U. L., Saibin, T. C., Labadin, J., & Aziz, N. A. (2018). Preliminary Investigation: Teachers´ Perception on Computational Thinking Concepts. Journal of Telecommunication, Electronic and Computer Engineering, 9(2-9), 23-29. https://jtec.utem.edu.my/jtec/article/view/2672

Liu, X., Wang, X., Xu, K., & Hu, X. (2023). Effect of Reverse Engineering Pedagogy on Primary School Students’ Computational Thinking Skills in STEM Learning Activities. Journal of Intelligence, 11(2), 36. https://doi.org/10.3390/jintelligence11020036

Lockwood, J., & Mooney, A. (2017). Computational Thinking in Education: Where does it fit? A systematic literacy review. https://doi.org/10.48550/arXiv.1703.07659

Lye, S. Y., & Koh, J. H. L. (2014). Review on teaching and learning and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior, 41, 51-61. https://doi.org/10.1016/j.chb.2014.09.012

Menon, D., Romero, M., & Viéville, T. (2019). Computational thinking development and assessment through tabletop escape games. International Journal of Serious Games, 6(4), 3-18. https://doi.org/10.17083/ijsg.v6i4.319

Merino-Armero, J. M., González-Calero, J. A., & Cózar-Gutiérrez, R. (2021). Computational thinking in K-12 education. An insight through meta-analysis. Journal of Research on Technology in Education, 54(3), 1-26. https://doi.org/10.1080/15391523.2020.1870250

Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for integrating technology in teacher knowledge. Teachers College Record, 108(6), 1017-1054. https://doi.org/10.1111/j.1467-9620.2006.00684.x

Molina-Ayuso, A., Adamuz, N., López, R. B., & Torralbo, M. (2022). Introduction to Computational Thinking with Scratch for Teacher Training for Spanish Primary School Teachers in Mathematics. Education Sciences, 12(12), 899. https://doi.org/10.3390/educsci12120899

Moreno-León, J., Robles, G., Román-González, M., & Rodríguez, J. D. (2019). No es lo mismo: un análisis de red de texto sobre definiciones de pensamiento computacional para estudiar su relación con la programación informática. Revista Interuniversitaria de Investigación en Tecnología Educativa (RIITE), (7), 26-35. https://doi.org/10.6018/riite.397151

Moreno-León, J., Robles, G., & Román-González, M. (2015). Dr. Scratch: Análisis Automático de Proyectos Scratch para Evaluar y Fomentar el Pensamiento Computacional. Revista de Educación a Distancia (RED), 46(10). https://doi.org/10.6018/red/46/10

Noh, J., & Lee, J. (2019). Effects of robotics programming on the computational thinking and creativity of elementary school students. Education Technology Research and Development, 68(1), 463-484. https://doi.org/10.1007/s11423-019-09708-w

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanvill, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., Mcdonald, S., McGuinness, L. A., Stewart, L. A., Thomas, J., Tricco, A. C., …, & Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Bmj, 372, 71. https://doi.org/10.1136/bmj.n71

Palts, T., & Pedaste, M. (2020). A Model for Developing Computational Thinking Skills. Informatics in Education, 19(1), 113-128. https://doi.org/10.15388/infedu.2020.06

Park, Y.-S., & Park, M. (2018). Exploring Students Competencies to be Creative Problem Solvers With Computational Thinking Practices. Journal of the Korean Earth Science Society, 39(4), 388-400. https://doi.org/10.5467/JKESS.2018.39.4.388

Pollak, M., & Ebner, M. (2019). The Missing Link to Computational Thinking. Future Internet, 11(12), 263. https://doi.org/10.3390/fi11120263

Popat, S., & Starkey, L. (2018). Learning to code or coding to learn? A systematic review. Computers & Education, 128, 365-376. https://doi.org/10.1016/j.compedu.2018.10.005

Rich, P., Mason, S., & O´Leary, J. (2021). Measuring the effect of continuous professional development on elementary teachers’ self-efficacy to teach coding and computational thinking. Computers & Education, 168(3), 104196. https://doi.org/10.1016/j.compedu.2021.104196

Rich, P., & Langton, M. B. (2016). Computational Thinking: Toward a Unifying Definition. In J. M. Spector, D. Ifenthaler, D. G. Sampson, & P. Isaias (Eds.), Competencies in Teaching, Learning and Educational Leadership in the Digital Age (pp. 229-242). Springer. https://doi.org/10.1007/978-3-319-30295-9_14

Román-González, M., Moreno-León, J., & Robles, G. (13-15 de julio de 2017). Complementary Tools for Computational Thinking Assessment. International Conference on Computational Thinking Education 2017. Hong-Kong, China. https://www.researchgate.net/publication/318469859_Complementary_Tools_for_Computational_Thinking_Assessment

Román-González, M., Pérez-González, J. C., & Jiménez-Fernández, C. (18-20 de octubre de 2015). Test de Pensamiento Computacional: diseño y psicometría general. III Congreso Internacional sobre Aprendizaje, Innovación y Competitividad (CINAIC 2015). Madrid, Spain. https://doi.org/10.13140/RG.2.1.3056.5521

Sáez, J. M., & Cózar, R. (2017). Pensamiento computacional y programación visual por bloques en el aula de Primaria. Educar, 53(1), 129-146. https://doi.org/10.5565/rev/educar.841

Sánchez-Meca, J. (2022). Revisiones sistemáticas y meta-análisis en Educación: Un tutorial. Revista Interuniversitaria de Investigación en Tecnología Educativa (RIITE), 13, 5-40. https://doi.org/10.6018/riite.545451

Segredo, E., Miranda, G., & León, C. (2017). Hacia la educación del futuro: El pensamiento computacional como mecanismo de aprendizaje generativo. Education in the Knowledge Society (EKS), 18(2), 33-58. https://doi.org/10.14201/eks2017182335

Sengupta, P., Kinnebrew, J. S., Basu, S., Biswas, G., & Clark, D. (2013). Integrating computational thinking with K-12 science education using agent-based computation: A theoretical framework. Education and Information Technologies, 18(2), 351-380. https://doi.org/10.1007/s10639-012-9240-x

Serrano, J. L., & Ortuño, G. (2021). Percepciones del profesorado en formación sobre el desarrollo del pensamiento computacional desde el Modelo 5PC. EDUTEC. Revista Electrónica de Tecnología Educativa, (78), 212-230. https://doi.org/10.21556/edutec.2021.78.2173

Shen, J., Chen, G., Barth-Cohen, L., Jiang, S., & Eltoukhy, M. (2020). Connecting computational thinking in everyday reasoning and programming for elementary school students. Journal of Research on Technology in Education, 54(2), 205-225. https://doi.org/10.1080/15391523.2020.1834474

Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017). Demystifying computational thinking. Educational Research Review, 22(1), 142-158. https://doi.org/10.1016/j.edurev.2017.09.003

Sun, L., Hu, L., & Zhou, D. (2021). Which way of design programming activities is more effective to promote K‐12 students’ computational thinking skills? A meta-analysis. Journal of Computer Assisted Learning, 37(4), 1048-1062. https://doi.org/10.1111/jcal.12545

Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020). Assessing computational thinking: A systematic review of empirical studies. Computers & Education, 148(1), 103798. https://doi.org/10.1016/j.compedu.2019.103798

Tran, Y. (2019). Computational Thinking Equity in Elementary Classrooms: What Third-Grade Students Know and Can Do. Journal of Educational Computing, 57(1), 3-31. https://doi.org/10.1177/0735633117743918

Tsai, M., Liang, J., Lee, S., & Hsu, C. (2021). Structural Validación for the Development Model of Computational Thinking. Journal of Educational Computing Research, 60(1), 56-73. https://doi.org/10.1177/07356331211017794

Uslu, N. A., Yavuz, G. O., & Usluel, Y. K. (2022). A systematic review study on educational robotics and robots. Interactive Learning Environments, 1-25. https://doi.org/10.1080/10494820.2021.2023890

Voogt, J., Fisser, P., Good, J., Mishra, P., & Yadav, A. (2015). Computational thinking in compulsory education: Towards an agenda for research and practice. Education and Information Technologies, 20(4), 715-728. https://doi.org/10.1007/s10639-015-9412-6

Voon, X., Wong, S., Wong, L, Khambari, M., & Syed-Abdullah, S. (2023). Developing pre-service teachers’ computational thinking through experiential learning: hybridization of plugged and unplugged approaches. Research and Practice in Technology Enhanced Learning, 18(6), 1-27. https://doi.org/10.58459/rptel.2023.18006

Wei, X., Lin, L., Meng, N., Tan, W., Kong, S.-C., & Kinshuk. (2021). The effectiveness of partial pair programming on elementary school students’ Computational Thinking skills and self-efficacy. Computers & Education, 160, 104023. https://doi.org/10.1016/j.compedu.2020.104023

Weigend, M., Vaníček, J., Pluhár, Z., & Pesek, I. (2019). Computational Thinking Education through Creative Unplugged Activities. International Olympiads in Informatics Journal (IOI), 13, 171-192. https://doi.org/10.15388/ioi.2019.11

Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33-35. https://doi.org/10.1145/1118178.1118215

Xia, L., & Zhong, B. (2018). A systematic review on teaching and learning robotics content knowledge in K-12. Computers & Education, 127, 267-282. https://doi.org/10.1016/j.compedu.2018.09.007

Yadav, A., & Chakraborty, P. (2023). Introducing schoolchildren to computational thinking using smartphone apps: A way to encourage enrolment in engineering education. Computer Applications in Engineering Education, 1-19. https://doi.org/10.1002/cae.22609

Yadav, A., Gretter, S., Good, J., & McLean, T. (2017). Computational Thinking in Teacher Education en P. Rich and C.B. Hodges (Eds.), Emerging Research, Practice, and Policy on Computational Thinking (pp. 205-220). Springer. https://doi.org/10.1007/978-3-319-52691-1_13

Yadav, A., Krist, C., Good, J., & Caeli, E. N. (2018). Computational thinking in elementary classrooms: Measuring teacher understanding of computational ideas for teaching science. Computer Science Education, 28(4), 371-400. https://doi.org/10.1080/08993408.2018.1560550

Yadav, A., Mayfield, C., Zhou, N., Hambrusch, S., & Korb, J. T. (2014). Computational Thinking in Elementary and Secondary Teacher Education. ACM Transactions on Computing Education, 14(1), 1-16. https://doi.org/10.1145/2576872

Zapata-Ros, M. (2015). Pensamiento computacional: Una nueva alfabetización digital. Revista de Educación a Distancia (RED), 46(4), 1-47. https://doi.org/10.6018/red/46/4

Zhang, L., & Nouri, J. (2019). A systematic review of learning by computational thinking through Scratch in K-9. Computers & Education, 141, 103607. https://doi.org/10.1016/j.compedu.2019.103607

Notes