INTRODUCTION

Digital and networked technologies have become so deeply integrated into work, education, and everyday life that categorizing specific activities as "digital" has lost meaning. To address this evolving socio-technical landscape, the term "postdigital" has gained popularity, acknowledging the messy and unpredictable intersections between the digital and the analogue, the technological and the non-technological, the biological and the informational, and the old and the new (Jandrić et al., 2022; Taffel, 2016).

In this postdigital age, the generation, processing, circulation, and commodification of data resulting from our daily lives have become more pervasive than ever. Data, which can be defined as measurements or observations collected for information purposes (Australian Bureau of Statistics, 2022), is constantly being collected by our surroundings, the places we visit, the online services and devices we use, and is often used to make inferences about our behaviour through algorithms. Monetizing data extracted through various forms of surveillance and monitoring has become a central aspect of the current economy (Bloom, 2019; Zuboff, 2019) with data serving as the new oil and data refineries playing a crucial role in the production of economic value within the framework of informational capitalism (Cohen, 2019).

However, our relationship with data is not merely passive, as we also consume vast amounts of data that shape our worldview and influence our decisions. Therefore, there is a growing need to educate people about reading, understanding, and analyzing data (Raffaghelli & Stewart, 2020). Furthermore, individuals may also want to explore patterns within the data, bring about changes, and communicate ideas using data. Consequently, the ability to effectively and ethically handle data, as well as navigate data-driven realities, has become essential for work, education, and active citizenship in contemporary societies (Markham & Pereira, 2019; Nguyen, 2021). Moreover, data literacy is not only critical for personal and professional growth but also plays a crucial role in promoting social justice (Atenas et al., 2020; Louie et al., 2022). Empowering individuals with data literacy skills allows them to critically assess information, challenge biases, and identify inequities in data-driven decision-making processes (Markham, 2020). However, achieving data literacy as a fundamental part of digital education for citizens has proven to be a complex challenge (Marín & Castañeda, 2022; Pangrazio & Selwyn, 2019).

Against this backdrop, Data Literacy for Citizenship (DALI) ¹ emerged as a European Erasmus+ project that aims to empower adults for responsible citizenship and civic engagement in a postdigital world, by supporting the development of key competencies related to the use of data and the understanding of the associated implications. The project targets three demographic groups (young adults, general adults, and seniors) across four countries (Germany, Norway, Spain, and the UK). In addition to the impact of data on adults' lives, they also bear responsibility for regulating how young people engage with data both at home and in educational settings, as adults are typically legally responsible for their children's data in most countries.

DALI focuses on co-creating, piloting, and evaluating pedagogical strategies, as well as developing games and playful learning resources for adult learners. It also involves engaging stakeholders in the field of adult education to facilitate implementation. The project adopts a playful approach to increase learning demand and participation through effective outreach, guidance, and motivational strategies (Arnab et al., 2019; Whitton, 2018).

With this process in mind, DALI began its work by creating a framework that helps define the field and the boundaries of data literacy (from here on DL) as a way of empowering people in the digital world and strengthening their agency.

There have been many attempts to approach an ethical view of data through frameworks (a great analysis of them in Atenas et al., 2023), and some specific attempts have already been made to define DL previously. For example, DL has been incorporated into frameworks that outline essential areas of competence encompassing knowledge, skills, and attitudes necessary for Digital Literacy (Marín & Castañeda, 2022). The European Commission's Digital Competence Framework for Citizens (DigComp) includes Information and Data Literacy as one of its five competency areas (Carretero Gomez et al., 2017; Kluzer & Priego, 2018). However, in certain work contexts, such as teaching, only specific aspects of DL are covered in the European Commission's DigCompEdu, primarily related to “responsible use” (Caena & Redecker, 2019). Additionally, while DigComp offers a broader perspective on digital competencies, a dedicated Data Literacy framework is necessary to focus on specialized skills for data analysis, interpretation, and ethical engagement. This new framework would empower individuals for data-driven decision-making and foster social justice through equitable data practices.

Different data literacy frameworks exist, including Learn2Analyse, Open Data Institute, and Data Citizenship Framework (Yates et al., 2021). They vary in aspects like target audience and creation methodology (Table 1).

Various frameworks incorporate ethics to some extent, primarily concerning the use of data in their respective professional contexts. While these frameworks offer valuable contents, their creation processes lack detailed explanations, except for the Educational Data Literacy Competence Framework, which is based on an expert-based survey method (Yates et al., 2021). Conversely, the Data Citizenship Framework's development is inadequately described, with authors primarily from the United Kingdom, except for one person affiliated with a Chinese university. The creation process of the Data Skills Framework remains unknown.

To address this gap and promote interdisciplinary and cross-cultural interpretations, the DALI project employed a co-creation process to develop its own framework. This approach allowed for collective reflection on DL while acknowledging the theoretical and ideological diversity of the project's professionals. The framework creation utilized a Delphi study, recognizing the complexity of the Data Literacy Phenomenon (Marín & Castañeda, 2022). Additionally, the DALI framework aligns with the European Union's 2018 definition of Digital Literacy (European Union, 2018).

In addition, the basic structure for the framework is derived from the DigComp structure. DigComp is structured into the following components: a) competence areas (elements); b) competences titles and descriptors (sub-element, sub-competence), c) proficiency levels and d) examples of use (Carretero Gomez et al., 2017). Like the DigComp structure, the resulting framework in this study will also include indicators for precisely three levels of development (proficiency levels) for each sub-competence in each element or competence area (levels A to C from basic to advanced proficiency).

This article presents the collective process we followed with the aim of defining the DALI Data Literacy Framework for critical citizenry that subsequently underpins the design of the game-based networked learning strategies and playful resources generated during the project.

METHOD

The study's main objective was to collaboratively develop a framework that would be the foundation for the rest of the DALI project, a framework that is developed and implemented with indicators of the desirable basic, intermediate, and advanced levels of performance for the target audience of the project (i.e. adults). The DALI Data Literacy Framework (from here on the DALI Framework) emerged from a collective expert construction, rather than from a literature review. Therefore, we followed the Delphi technique in order to identify the essential areas that should be included in our framework, and we used the technique itself for its development (Okoli & Pawlowski, 2004).

A Delphi Study

Several circumstances led us to use the Delphi method. First, it is useful for creating a participatory synthesis, or what is the same, collecting, analysing and building opinions from homogeneous or heterogeneous groups of researchers to “illuminate certain aspects of a phenomenon or build collective understanding regarding this” (Cohen et al., 2017, p. 463). In addition, it has been extensively adopted in different fields, such as social and environmental sciences, to engage experts in dialogue and interaction around an object of study (Bond et al., 2021; Fefer et al., 2016). Particularly, it has been important in the field of educational research (Guàrdia et al., 2022). The Delphi technique can be understood as “a group communication process as well as a method of achieving a consensus opinion” (Salkind, 2010, p. 343), in such a way that allows participants to effectively “deal with a complex problem” (Linstone & Turoff, 1975, p. 3). It is worth noting that the Delphi technique is aligned with the co-creation and co-design approach under which the whole work development in the DALI project is carried out since they are also addressed to “enable organisations, groups and individuals to interact, collaborate and solve problems by jointly generating solutions and creating value” (El-Jarn & Southern, 2020, p. 192).

Considering the definition of the DALI Framework as the first task for the project, this Delphi aimed to extract the ideological statements related to DL from within the core group, as well as to structure a group communication process that would take advantage of the academic diversity of the group for achieving a broader perspective related to the study object, and has been considered reliable to develop frameworks (Chen, 2019; Fefer et al., 2016). Nevertheless, a Delphi study has five conditions (1) a consultation of a mature set of experts –probably the essential characteristic of the technique– (2) anonymous –none of them should know the specific inputs of the others– (3) multiple rounds –which should be configured as iterations of the process–, (4) with feedback of the results and (5) the opportunity for the participants to reconsider their position (Becuwe et al., 2017; Cabero, 2013).

Panellists

The selection of experts is critical for the final validity of the results (Seo et al., 2020). In this case, the experts were chosen by the project partners. Each partner university chose two experts to take part in the Delphi panel, considering not just the role of the partner in the specialized consortium, but also the expertise of each participant. Ten experts in education, from five different institutions, in four other countries (Germany, Norway, Spain and the United Kingdom) and with diverse backgrounds and perspectives on the use and study of technology for education participated in the process. Nine of them held a PhD on topics relevant to education and technology.

The Delphi Structure

Taking advantage of the flexibility of the Delphi approach and the possibility of making some adaptations to it – including preliminary literature reviews which have been also recurrent in education studies (Seo et al., 2020) –, in this study, we decided to implement the Delphi panel following Bond et al.’s (2021) approach in a three and a half rounds’ version (see Figure 1) which take 20 weeks in total.

The Delphi structure aimed to collaboratively shape the DALI Framework through iterative questionnaires and controlled opinion feedback (Dalkey & Helmer, 1963). Each iteration began with open-ended questions and an explanation phase (Chen, 2019; Fefer et al., 2016), where panellists individually addressed specific aspects of the framework. The moderator, representing the University of Murcia, collected, anonymized, and organized the responses for discussion in synchronous sessions. During these sessions, discussions occurred in small groups (2-3 people) followed by presentation and consensus-building in the larger group (Bond et al., 2021).

The work rounds served as phases for deepening the framework, with each round building upon the previous discussions. Consequently, the results of each round were directly integrated into the ongoing work. This approach effectively captured diverse perspectives from geographically dispersed panelists while ensuring anonymity to mitigate dominance or external social dynamics (Fake & Dabbagh, 2021; Fefer et al., 2016).

Data Collected

This section describes the data collected as part of the Delphi process in each one of the rounds depicted in Figure 1.

Round 1

As can be seen in Figure 1, the first round of the process focused on defining DL based on participants' backgrounds and the most relevant DL-related paper they could refer to. The panellists also identified key elements for the DALI Framework. During the online synchronous session, discussions centered on problematic terms and abilities in the DL definition. Additionally, priority order, relevance of terms and elements, and elimination of irrelevant information were discussed.

In this first round, panellists were asked about their preferred definition and elements for shaping DL. Results were organized in a definition and a list of elements. In addition, for representation purposes, the definitions were visualized using a word/tag cloud technique (McNaught & Lam, 2010) that highlighted the most repeated terms included in the definition (see Figure 2).

The definition of DL was based on nine scientific/academic articles highly relevant to the panellists. Three of these papers were used as references for defining the elements of DL. Additionally, the panellists were asked to include four essential references on DL, resulting in 19 papers included in Appendix 1.

From these data, it is evident which papers were particularly influential in the framework creation (Calzada-Prado & Marzal, 2013; Pangrazio & Selwyn, 2019; Raffaghelli, 2019). Moreover, it seems interesting that from the 39 different authors contributing to these papers, the work of Raffaghelli – which appears five times in three different papers – seems very influential in the development process. Also, the contribution of Carmi and Yates – which appears in two different papers in 2020 – is referenced twice by the panellists. Twelve authors were referenced by panellists at least twice: Cavero, Gooch, Kortuem, Lockley, Mandinach, Maybee, Montaner, Pawluczuk, Rashid, Schüller, Wolff, and Zilinski.

Round 2

The second round focused on the basic elements of DL and their relationship with DigComp elements. During the online synchronous discussion, the perceived connections between the defined DALI framework elements and DigComp elements (information and data literacy, communication and collaboration, digital content creation, safety, and problem-solving) were examined. The group also deliberated on better naming for each element and the inclusion of relevant skills. Part of the reflection involved rating the relationship between each framework element and DigComp elements on an 8-Likert scale, ranging from "Not related at all" (0) to "Perfectly related" (7). The results are depicted in Figure 3.

The findings suggest that some elements from DigComp, such as “Safety” (e.g., sub-element "protecting personal data and privacy") and “Information & Data Literacy” (e.g. sub-element "managing data, information and digital content") are transversal to the three elements of DALI (understanding data, acting on data and engaging through data), whereas other elements, like “Communication & Collaboration” (e.g., sub-element "engaging in citizenship through digital technologies") or “Problem-Solving” (e.g., sub-element "creatively using digital technologies") are more present in some aspects of DL (e.g., engaging through data).

Round 3

In the third round, the focus was on defining the skills for each DL element and identifying them through indicators in three proficiency levels. Online synchronous discussions centred on validating the coherence of the skill levels. The process involved panellists filling a grid with sub-competences and elements, along with three columns (level A, B, and C) containing indicators. After anonymizing and compiling the responses, participants reviewed the components in groups and discussed the final wording of each indicator.

Final Round

A final half-round was used to discuss the final version of the DALI framework. The group coordinator made a first draft of the complete framework, based on the data collected during the previous rounds. This final half-round skipped the individual feedback -apart from typos and grammar precisions– and went directly to the group work discussion to elicit general impressions about the framework.

RESULTS: THE DALI FRAMEWORK

We now present the Delphi process's final product by describing the DALI Framework. Following an organizational structure like the EU DigComp framework, it includes a DL definition, the DL elements, the framework indicators, and levels of performance.

DL Definition

DL is about how people might understand, use, and engage with the data encountered in everyday life for the citizenry. It implies finding ways to make informed decisions – in everyday life and in various contexts according to personal or collective goals. To ask and answer questions from data sets through an inquiry process, considering ethical use of data.

DL includes an understanding of data, and awareness/attitudes toward non-neutrality/biased data (collection, etc.). It implies knowledge about collecting, selecting, storing, preserving, and managing data; analysing, evaluating, interpreting, critiquing, applying, using, and working with data; and representing, visualizing, and communicating stories from data.

DL also includes skills for making critical judgments and interrogating the claims accompanying data systems, including ethical and legal aspects that affect themselves and other people’s rights. It also comprises abilities to use data as part of a design process as solving problems and making decisions for different purposes.

The DALI Framework reconciles all these considerations into four main elements. Three of them are mainly independent of each other, even though in some cases the boundaries between them overlap: (1) Understanding Data, (2) Acting on Data, and (3) Engaging Through Data. An extra element is considered transversal to the others: (4) Ethics & Privacy. While it is present in (1), (2) and (3) in terms of content and indicators, it is a fundamental element in all of them.

Some crucial points should be emphasized in the development of the definition and its components. In the process of defining the term, a significant discussion arose regarding citizenship versus citizenry, and we ultimately chose the latter term to be more inclusive, participatory, and justice-oriented (Westheimer & Kahne, 2004). This decision allows for a broader perspective, moving beyond the narrow confines of official citizenship status in local contexts. Also, concerning the DALI elements, the choice of names for each of them was not straightforward and without discussion. On the contrary, panellists started focusing on data for the components’ titles. However, during the discussion sessions, an important point was made to give more importance to the people and the actions they develop with data. Therefore, the participants ended up entitling the DALI components from a participant’s action perspective rather than from a data perspective.

DL Elements

The DALI Framework is represented in Figure 4 with its elements (mentioned before in the DL definition) and sub-elements.

All four elements are extensive and complex, so their definition is challenging. Nevertheless, for this project, we are defining the boundaries of them by the description of the contents that each one of them includes:

Understanding Data

Understanding data refers to cognitive operations, reflections, and processes that do not imply necessarily actions:

• Understanding data as a representation of reality, not the reality itself. What data is, what form it takes, and how it can be used in society (including personal data, institutional data, etc.).
• Understanding where data comes from, i.e., data origins (e.g., sensors, own apps, use of a tool, GPS on the own phone, etc.).
• Understanding different types (e.g., sensor data, audio data), and technical formats (e.g., MP4, JPG), as well as how these data can be collected from different environments.
• Understanding the complexity of data: big vs. small data, variations in complexity of data.
• Knowing about technical/technological pre-conditions for data creation and use (connecting devices, deciding on device settings…).
• Data processing and data manipulation (e.g., understanding that apps use data, algorithms use data).
• The potential and drawbacks of big data in different realms of society, such as health, education, economics, security, etc. (e.g., trade-offs of using social media, data surveillance).
• The human-data relationship: When to use automated processes vs human actions. Who makes the decision?
• Data as persistent and potentially stored: Data Security; data surveillance, opportunities with data (weather, maps, etc.); my data.
• The Data tools: how tools work not from the user perspective but the viewer perspective - Targeted advertisements. Identifying misrepresentation of data.
• How to identify, locate and use data sets, public data, databases, and APIs.
• Question data and use of data. How data is monetised, for which purposes it is being collected -- “data as the new oil.”

Consequently, this element integrates three sub-elements: (1) Knowledge, (2) Awareness, and (3) Critical thinking.

Acting on Data

Acting on data explicitly refers to actions to be carried out on data, such as:

• Organising data.
  • Synthesizing, visualizing, and representing data in different formats. “Translate” data into language.
  • Clarifying: It should not be about doing one’s own analyses but about transmitting information via data to others. It might be that oneself did not necessarily conduct such analyses.
  • Identifying misrepresentation of data.
• Using data to change one’s own behaviour.
• Become aware of one’s own role for acting on data from the different perspectives of a citizen (professional, familiar, etc.).
• Processing, protecting and storage (personal data management), how to move data from one application to another.
• Adapting to new scenarios by changing one’s own choices and practices based on new situations (configuring privacy aspects, revoking access, requesting to have one’s own data erased…).
• Putting data rights in practice.
• Making informed decisions when interacting with data-collecting actors (e.g., mobile apps, internet portals, employers…).
• Interacting with key stakeholders (e.g., data protection agencies) as needed for the resolution of certain situations.

Consequently, this element includes three sub-elements that help us to define it: (1) Collecting data, (2) Managing data, and (3) Sharing data artifacts.

Engaging Through Data

In this element, the ambition goes further to actions that affect individuals and the world. How do we engage through data? (Individual (I) & Collective (C))

• Self-regulate own data footprint (I)
• Taking own decisions based on critical consideration of data (personal, professional...) (I)
• Communicating data meaning to stakeholders or to other peers (C)
• Using data as a basis or activism for data engagement (C)
• Raising collective data awareness (C)
• Adapting data (I & C)
• Participating in data-based policy-making processes (C)
• Understanding balance between individual and social benefits and risks related to data literacy (I & C)

In consequence, this element is composed by four sub-elements: (1) Policy and regulation, (2) Taking decision, (3) Data activism, and (4) Data advocacy.

Ethics & Privacy

At this point, the fourth element, Ethics & Privacy, must be considered, as these concepts are omnipresent in the other three, as can be seen in Figure 4.

Ethics and Privacy are the base to build any component of this framework, and this ethical perspective must underlie all the skills and levels considered. Some key concepts of privacy include the relationship between personal information and data, the right for individuals to control their own data or cybersecurity concerns. In addition, key concepts of ethics include that some actions on data might be ethically questionable, despite being lawful, or algorithmic biases.

Therefore, Ethics and Privacy must be taken into account in a cross-cutting manner in the three main elements of the DALI Framework.

Framework Indicators and Levels

To make this framework operational, and to mirror other frameworks developed worldwide (e.g., DigComp), we have identified which indicators better shape the different elements and sub-elements in progressive levels of expertise, from level A (the most basic one) to level C (the advanced one).

In Table 2, we can see the correspondence between elements, sub-elements, and levels, with their descriptions.

LIMITATIONS

There are two contextual limitations that must be acknowledged in this study. Firstly, the Delphi study participants were university staff from Europe, mainly representing WEIRD countries (Western, Educated, Industrialized, Rich, and Democratic) ( Henrich et al., 2010). Although efforts were made to include diverse scholarly backgrounds, all the participants characteristics remained largely common to these global north contexts. Secondly, the framework's subject matter is closely tied to the current technological development in these countries.

References provided by the panellists during data collection exhibit a clear trend of citation in one language and publication type (Macgilchrist et al., 2022), with a notable absence of publications from literacy journals and publications in languages other than English, resulting in limited influence from less mainstream discourses.

These limitations underscore that the framework is not universally applicable. However, it is proposed as a starting point for adaptation and validation within the project and through external research endeavours. Validation should encompass diverse contexts to ensure effectiveness on fostering human agency, as defined by Jääskelä et al. (2017), and influenced by contextual singularities.

Rethinking and reformulating the framework in new contexts aims to enhance relevance and incorporate diverse academic perspectives. Furthermore, the framework is designed to align with ongoing technical, social, and organizational changes, requiring regular implementation. Its application contributes to continuous updates, akin to other international framework proposals (Kluzer & Priego, 2018). The DALI framework is conceived as a flexible and adaptable outcome, with the goal of long-term usefulness.

DISCUSSION AND CONCLUSIONS

Derived from the Delphi method applied in this work, we developed a framework for DL consistent with the target group of the DALI project (i.e., adults). Drawing on other recognised frameworks in the field of digital literacy as points of reference, we designed the DALI Data Literacy Framework to be in line with, and complement, already existing digital frameworks by extending the notion of data literacy – as a specific competence that must be included– and considering the needs of the adult citizenry in this respect. This is especially noted in work and results of the second round, where different DigComp areas were considered to integrate aspects of DL.

The DALI Data Literacy Framework includes both a definition and a description of four elements, from which three of them are structured into sub-elements with levels of performance, and the fourth has sub-elements embedded across the whole model. The DALI definition of data literacy is in line with early definitions collected by Wolff et al. (2016) in the sense that it aspires to help inform decisions and support problem-solving, and a granular approach by including the diverse subset of skills and abilities that are covered under the concept. Also, it is aligned with recent definitions, which integrate the idea of data literacy for social justice. This underpinning approach in the DALI definition is supported by the perspective of the global citizen included under the term “citizenry”, which goes beyond the idea of the active citizen in local contexts and assumes global activism for the good (Chowdhury et al., 2019; Sutton, 2008).

Regarding the four elements there is a clear progressive relationship between the three main elements. The first element, Understanding data, covers the conceptual level of the framework and includes two declarative sub-elements mainly related to knowing about the definition and being aware of its existence and practical implications. The third sub-element emerges as the unique approach to DL for the citizenry. It addresses the critical approach through which data literate citizens can think critically about data implications for people. Many citizens might already be aware of some of the core concepts within this element, such as the existence and the value of data in the current economic market but it focuses on the requirements for them to develop new cognition regarding data thinking and to comprehend the role of data in the world. While this element is aligned with the ‘Data Comprehension & Interpretation’ one from the Learn2analyze’s framework, in the DALI Framework, this element moves on to critical thinking. It is also noteworthy that within understanding data, there is the sub-element for data visualisation, which has been highlighted as troublesome at the compulsory educational level and, at the same time, an essential skill for informed citizens (Shreiner & Dykes, 2021).

The second element, Acting on data, involves higher-level actions on the data, such as collecting, managing, analysing, and sharing. It focuses on actions that citizens can perform on data and require different levels of competence, ranging from very simple skills, such as managing plain text or Excel files by manually inputting data, to complex data management processes utilising databases.

In this respect, Data from Eurostat on individuals' level of computer skills (2021) show that 57% of the EU citizens (27 countries) have copied or moved files between folders, devices or in the cloud, and 58% used word processing software in the last three months (other valuable data: the creation of files integrating different elements 38%; used spreadsheet software 38%; used advance features of spreadsheet software or organise, analyse and structure date 21%) (EU survey on the use of Information and Communication Technologies (ICT) in households and by individuals, annual survey)². It means most European individuals have basic skills regarding the collection, management, and sharing of plain data, including txt, xlsx, or PDF files, since these skills are generally needed as part of the essential digital competencies for performing many jobs or studies. However, some other levels, such as database management or pivoting tables, will only be available for specific professionals such as database administrators or data analysts. In this sense, only those citizens with some technical professional roles need to advance across the levels of expertise in this element.

The third element, Engaging through data, involves a deep conceptual understanding of data as well as knowledge of the actions that can be performed on data to enable taking decisions based on data, data activism, and advocacy, as well as understanding data-related policies and regulation. In this sense, the framework is conceptually aligned with the vision of citizenry, which focuses on the role of citizens to participate in social responsibilities and be engaged in local, national, and global issues (Chowdhury et al., 2019; Sutton, 2008). Additionally, the activism underpinning social change is closely related to the transformational implications of digital literacy in early work (Martin & Grudziecki, 2006). Therefore, this element implies developing higher-level cognitive processes and understanding the interplay between data and society -including ourselves as community individuals. These competencies go from straightforward day-to-day life decisions, such as optimising the selection of the data you will allow sharing after starting using a new mobile app (see, e.g., the Data Detox Kit³), to more collective endeavours, such as using X (former Twitter) or petition platforms to advocate for more restrictive regulation policies on geolocation tracking or targeted advertising. In this case, citizens need to learn how they can support efforts to protect the public interest by engaging in collective action by sharing or signing data-related petitions (e.g., those on Change.org) or taking part in activists’ actions and projects (e.g., Xnet⁴). This relies on understanding the underlying data issues addressed by the first element of the framework.

Finally, the last and cross-cutting element, Ethics & privacy, addresses aspects that are inherently present in all the other elements due to the datafied society in which we live. Contrary to other frameworks, such as Learn2analyze where ‘Data Ethics’ is considered as an element by itself, from the point of view of the panellists of the DALI framework, ethics and privacy are intrinsically present in every action, and thus, in every element, that we carry out with data. Thus, this element implies that citizens need to be aware of data’s ethical and privacy implications. At the most basic level, EU citizens have some awareness of privacy issues; for instance, according to Eurostat⁵ 72% of the population (across 27 countries) know that cookies can be used to monitor the behaviour of internet users. However, they have a more limited understanding of how the data collected from them is used by third parties or what they can do to prevent that from happening. Research has stated that only a small portion of young adults have developed skills to manage their online data by uploading positive data about themselves and at the same time devoting time to manage data uploaded by others (Lorenz & Kikkas, 2014). Similarly, most citizens would see that stealing data would be ethically wrong. Still, ethical aspects such as algorithmic biases due to historical and societal minority discriminations are complex to understand without the appropriate technical knowledge. Therefore, these aspects of the Ethics & privacy element are also heavily influenced by the knowledge levels of the other elements.

Against this backdrop, the framework presented here results from a participatory and collaborative process of research and creation, which is currently used to develop learning materials and curriculum innovation (Villatoro & de Benito, 2021). We understand that having operationalised this framework can result in a more realistic educational approach, in that the indicators can lead to clear learning goals, which help in the definition and development of didactic strategies for their development. Innovation in education is not an easy process (Fullan, 2007). While top-down developments have not been successful, developments by experts and field practitioners have improved the possibilities for innovation uptake.

As the digital literacy has been described as a dynamic and constantly evolving field (Pérez-Escoda et al., 2019), the same can be expected regarding data literacy, which simultaneously requires framework proposals that might be flexible to integrate such changes.

As pointed out earlier, the Delphi technique through which the panellists have arrived at a consensus for the DALI approach is aligned with the co-creation and co-design processes that inspire the whole DALI project development. Like the Delphi technique, these collaborative processes allow joint work across “time zones, spaces, disciplines and cultures” (El-Jarn & Southern, 2020, p. 192), which can increase the chances of a successful product. In this way, we envision the DALI framework as a successful product to be implemented and adapted in several different cultural backgrounds, being enriched in diverse epistemic or international contexts. We expect it will be valuable for designing future pedagogical experiences that can be relevant and “overcome cross-cultural differences” (Vespestad & Smørvik, 2020).

Since Gilster coined the term digital literacy in 1997, it has dramatically evolved into a complex concept that goes beyond technical skills (Marín & Castañeda, 2022; Meyers et al., 2013). The recent emergence of data literacy among the wide range of digital literacies is a new benchmark in the field. Some aspects allow us to suggest that the DALI framework is a valuable effort and contribution toward developing DL. First, regarding its structure, it is a model that balances a comprehensive and analytical envision of DL by describing a relatively limited number of sub-elements whose coherence is guaranteed by being grouped into elements that work both in a complementary and cross-cutting manner. Second, its holistic envisioning is due to the target group (adults) for whom it is conceived and overcomes other somewhat professional or technical descriptions. Third, the ethics approach, along with the data activism element, allows for overcoming individual approaches and leads the framework to a vision of data literacies for political and ideological activism and commitment toward the good of collectives.

The DALI Framework is not only flexible and scalable but also holds significant importance in teacher education and the creation of activities applicable to teaching. Teachers play a crucial role in shaping citizenship through critical methodologies in their respective disciplines, and the university has a fundamental responsibility in fostering conscientious citizenship. The adaptability of the DALI framework allows for modifications and the inclusion of new elements, making it suitable for various subpopulations within society. Its methodological strength stems from consensus reached through interdisciplinary discussions among experts in education and educational technologies. Unlike frameworks based solely on literature reviews, our work combines expert insights with existing literature, resulting in a more ideologically coherent framework. We hope that our DALI framework, along with its methodology, inspires other collectives to engage in dialogic processes and reflect on frameworks that contribute to a deeper understanding of the world (Atenas et al., 2023; Markham, 2020).

Furthermore, the framework can have multiple applications, for example, to help shape the needed data competencies of society, as a support framework for developing the curriculum of formal courses, or to profile the required data-related competencies for different professional roles in our community, e.g., teachers, who should develop data strategies while maintaining participatory educational practices (Kippers et al., 2018; Stewart, 2023).

The framework's subject matter is influenced by current technological and societal contexts, making it inherently changeable. Adaptation to specific constraints and realities of implementation is necessary, as the framework is not universally applicable. Regular updates are essential to align with evolving technological and social changes. Exploring the framework in local contexts and adapting it to different subpopulations present fruitful avenues for future research. Further research studies are needed to validate the framework's content, elements, and their relationships. Case studies within the project will confirm its adequacy for specific purposes and contribute to its continuous improvement. Adapting the framework to changing data needs over time is necessary (Markham, 2020; Sander, 2020). Including perspectives beyond the project consortium and from countries outside Europe would refine the framework further. This paper presents the initial DALI framework, which is an evolving artifact reflecting technological and data evolution. Future refinements are enabled by the framework's flexibility and scalability.

Current research in the DALI project is exploring the use of the framework for the co-design of the DALI games and preparing training actions according to the elements and sub-elements. Future work will consider how the framework may vary depending on the target group and the different levels. Finally, it could be very promising to develop instruments that provide evidence of DL skills, both from self-perception surveys and other performance tests.

Acknowledgments

The authorship team extend their heartfelt gratitude to all consortium members for their invaluable contributions to the development of the framework. This research has been supported by the Project DALI “Data Literacy for Citizenship, an ERASMUS+ Strategic Partnership (KA204-076492) funded by the European Union. Daniel Villar-Onrubia acknowledges the support of the ‘María Zambrano’ grant (RD 289/2021), funded by the European Union - Next GenerationEU programme. Victoria I. Marín acknowledges the support of the Grant RYC2019-028398-I funded by MCIN/AEI/ 10.13039/501100011033 and FSE “El FSE invierte en tu futuro”.

REFERENCES

Arnab, S., Clarke, S., & Morini, L. (2019). Co‑Creativity through Play and Game Design Thinking. Electronic Journal of E-Learning, 17(3), Article 3. https://doi.org/10.34190/JEL.17.3.002

Atenas, J., Havemann, L., & Timmermann, C. (2020). Critical literacies for a datafied society: Academic development and curriculum design in higher education. Research in Learning Technology, 28. https://doi.org/10.25304/rlt.v28.2468

Atenas, J., Havemann, L., & Timmermann, C. (2023). Reframing data ethics in research methods education: A pathway to critical data literacy. International Journal of Educational Technology in Higher Education, 20(1), 11. https://doi.org/10.1186/s41239-023-00380-y

Australian Bureau of Statistics. (2022). Statistical Language—What are Data? Australian Bureau of Statistics. https://www.abs.gov.au/websitedbs/D3310114.nsf/Home/Statistical+Language+-+what+are+data

Becuwe, H., Roblin, N. P., Tondeur, J., Thys, J., Castelein, E., & Voogt, J. (2017). Conditions for the successful implementation of teacher educator design teams for ICT integration: A Delphi study. Australasian Journal of Educational Technology, 33(2), 159-172. https://doi.org/10.14742/ajet.2789

Bloom, P. (2019). Monitored: Business and Surveillance in a Time of Big Data. Pluto Press.

Bond, M., Bedenlier, S., Marín, V. I., & Händel, M. (2021). Emergency remote teaching in higher education: Mapping the first global online semester. International Journal of Educational Technology in Higher Education, 18(1), 50. https://doi.org/10.1186/s41239-021-00282-x

Cabero, J. (2013). University teacher training in ICT. application of Delphi method for the selection of training content. Educación XX1, 17(1), 111-131. Scopus. https://doi.org/10.5944/educxx1.17.1.10707

Caena, F., & Redecker, C. (2019). Aligning teacher competence frameworks to 21st century challenges: The case for the European Digital Competence Framework for Educators (Digcompedu). European Journal of Education, 54(3), 356-369. https://doi.org/10.1111/ejed.12345

Calzada-Prado, J., & Marzal, M. Á. (2013). Incorporating Data Literacy into Information Literacy Programs: Core Competencies and Contents. Libri, 63(2), 123-134. https://doi.org/10.1515/libri-2013-0010

Carretero Gomez, S., Vuorikari, R., & Punie, Y. (2017). DigComp 2.1: The Digital Competence Framework for Citizens with eight proficiency levels and examples of use (Scientific Analysis or Review, Technical Guidance KJ-NA-28558-EN-N (online), KJ-NA-28558-EN-C (print), KJ-NA-28558-EN-E (ePub)). Publications Office of the European Union. https://doi.org/10.2760/38842

Chen, Y. (2019). Exploring Design Guidelines of Using User-Centered Design in Gamification Development: A Delphi Study. International Journal of Human–Computer Interaction, 35(13), 1170-1181. https://doi.org/10.1080/10447318.2018.1514823

Chowdhury, T., Holbrook, J., & Rannikmäe, M. (2019). Teachers’ Ownership Towards Using Socio-Scientific Issues for an Active Informed Citizenry. EDULEARN19 Proceedings, 3834-3840. https://doi.org/10.21125/edulearn.2019.0989

Cohen, J. E. (2019). Between Truth and Power: The Legal Constructions of Informational Capitalism. Oxford University Press. https://doi.org/10.1093/oso/9780190246693.001.0001

Cohen, L., Manion, L., & Morrison, K. (2017). Research Methods in Education (Edición: 8). Routledge. https://doi.org/10.4324/9781315456539

Dalkey, N., & Helmer, O. (1963). An Experimental Application of the DELPHI Method to the Use of Experts. Management Science, 9(3), 458–467. https://doi.org/10.1287/mnsc.9.3.458

El-Jarn, H., & Southern, G. (2020). Can co-creation in extended reality technologies facilitate the design process? Journal of Work-Applied Management, 12(2), 191-205. https://doi.org/10.1108/JWAM-04-2020-0022

European Union. (2018). Key Competences for Lifelong Learning (p. 104) [Proposal for a Council Recommendation]. European Commission.

Fake, H., & Dabbagh, N. (2021). The Personalized Learning Interaction Framework: Expert Perspectives on How to Apply Dimensions of Personalized Learning to Workforce Training and Development Programs. Ninth International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM’21), 501-509. https://doi.org/10.1145/3486011.3486503

Fefer, J. P., De-Urioste Stone, S., Daigle, J., & Silka, L. (2016). Using the Delphi Technique to Identify Key Elements for Effective and Sustainable Visitor Use Planning Frameworks. SAGE Open, 6(2), 2158244016643141. https://doi.org/10.1177/2158244016643141

Fullan, M. (2007). The new meaning of educational change. Teachers College Press.

Guàrdia, L., Romero, M., & Raffaghelli, J. E. (2022). Desarrollo Profesional Docente más allá de la pandemia: Un estudio Delphi sobre el potencial del concepto de ecologías de aprendizaje. Educación, 31(60), Article 60. https://doi.org/10.18800/educacion.202201.004

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2-3), 61-83. https://doi.org/10.1017/S0140525X0999152X

Jääskelä, P., Poikkeus, A.-M., Vasalampi, K., Valleala, U. M., & Rasku-Puttonen, H. (2017). Assessing agency of university students: Validation of the AUS Scale. Studies in Higher Education, 42(11), 2061-2079. https://doi.org/10.1080/03075079.2015.1130693

Jandrić, P., MacKenzie, A., & Knox, J. (2022). Postdigital Research: Genealogies, Challenges, and Future Perspectives. Postdigital Science and Education. https://doi.org/10.1007/978-3-030-97262-2

Kippers, W. B., Poortman, C. L., Schildkamp, K., & Visscher, A. J. (2018). Data literacy: What do educators learn and struggle with during a data use intervention? Studies in Educational Evaluation, 56, 21-31. https://doi.org/10.1016/j.stueduc.2017.11.001

Kluzer, S., & Priego, L. P. (2018). DigComp into action: Get inspired, make it happen. A user guide to the European Digital Competence Framework (JRC110624). Joint Research Centre (Seville site). https://publications.jrc.ec.europa.eu/repository/handle/JRC110624

Linstone, H. A., & Turoff, M. (1975). Delphi Method: Techniques and Applications. Addison-Wesley Educational Publishers Inc.

Lorenz, B., & Kikkas, K. (2014). If I Do Not Like Your Online Profile I Will Not Hire You! In P. Zaphiris & A. Ioannou (Eds.), Learning and Collaboration Technologies. Technology-Rich Environments for Learning and Collaboration (pp. 429-439). Springer International Publishing. https://doi.org/10.1007/978-3-319-07485-6_42

Louie, J., Stiles, J., Fagan, E., Chance, B., & Roy, S. (2022). Building toward Critical Data Literacy with Investigations of Income Inequality. Educational Technology & Society, 25(4), 142-163. https://www.jstor.org/stable/48695987

Macgilchrist, F., Potter, J., & Williamson, B. (2022). Reading internationally: If citing is a political practice, who are we reading and who are we citing? Learning, Media and Technology, 47(4), 407-412. https://doi.org/10.1080/17439884.2022.2140673

Marin, V. I., & Castañeda, L. (2022). Developing digital literacy for teaching and learning. In Handbook of Open, Distance and Digital Education (pp. 1-20). Springer. https://doi.org/10.1007/978-981-19-0351-9_64-1

Markham, A. N. (2020). Taking Data Literacy to the Streets: Critical Pedagogy in the Public Sphere. Qualitative Inquiry, 26(2), 227-237. https://doi.org/10.1177/1077800419859024

Markham, A. N., & Pereira, G. (2019). Analyzing public interventions through the lens of experimentalism: The case of the Museum of Random Memory. Digital Creativity, 30(4), 235-256. https://doi.org/10.1080/14626268.2019.1688838

Martin, A., & Grudziecki, J. (2006). DigEuLit: Concepts and Tools for Digital Literacy Development. Innovation in Teaching and Learning in Information and Computer Sciences, 5(4), 249-267. https://doi.org/10.11120/ital.2006.05040249

McNaught, C., & Lam, P. (2010). Using Wordle as a Supplementary Research Tool. The Qualitative Report, 15(3), 630-643. https://doi.org/10.46743/2160-3715/2010.1167

Meyers, E. M., Erickson, I., & Small, R. V. (2013). Digital literacy and informal learning environments: An introduction. Learning, Media and Technology, 38(4), 355-367. https://doi.org/10.1080/17439884.2013.783597

Nguyen, D. (2021). Mediatisation and datafication in the global COVID-19 pandemic: On the urgency of data literacy. Media International Australia, 178(1), 210-214. https://doi.org/10.1177/1329878X20947563

Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: An example, design considerations and applications. Information & Management, 42(1), 15-29. https://doi.org/10.1016/j.im.2003.11.002

Pangrazio, L., & Selwyn, N. (2019). ‘Personal data literacies’: A critical literacies approach to enhancing understandings of personal digital data. New Media & Society, 21(2), 419-437. https://doi.org/10.1177/1461444818799523

Pérez-Escoda, A., García-Ruiz, R., & Aguaded, I. (2019). Dimensions of digital literacy based on five models of development. Cultura y Educacion, 31(2), 232–266. Scopus. https://doi.org/10.1080/11356405.2019.1603274

Raffaghelli, J. E. (2019). Developing A Framework For Educators’ Data Literacy In The European Context: Proposal, Implications And Debate. EDULEARN19 Proceedings, 10520-10530. https://doi.org/10.21125/edulearn.2019.2655

Raffaghelli, J. E., & Stewart, B. (2020). Centering complexity in ‘educators’ data literacy’ to support future practices in faculty development: A systematic review of the literature. Teaching in Higher Education, 25(4), 435-455. https://doi.org/10.1080/13562517.2019.1696301

Salkind, N. J. (2010). Encyclopedia of Research Design. Sage Publications Sage CA: Thousand Oaks, CA. https://doi.org/10.4135/9781412961288

Sander, I. (2020). What is critical big data literacy and how can it be implemented? Internet Policy Review, 9(2). https://doi.org/10.14763/2020.2.1479

Seo, E., Ryu, J., & Hwang, S. (2020). Building key competencies into an environmental education curriculum using a modified Delphi approach in South Korea. Environmental Education Research, 26(6), 890-914. https://doi.org/10.1080/13504622.2020.1733493

Shreiner, T. L., & Dykes, B. M. (2021). Visualizing the teaching of data visualizations in social studies: A study of teachers’ data literacy practices, beliefs, and knowledge. Theory & Research in Social Education, 49(2), 262-306. https://doi.org/10.1080/00933104.2020.1850382

Stewart, B. E. (2023). The problem of the web: Can we prioritize both participatory practices and privacy? Contemporary Educational Technology, 15(1), ep402. https://doi.org/10.30935/cedtech/12668

Sutton, M. (2008). Knowledge citizenship for active informed citizenship. South African Journal of Information Management, 10(4), 5. https://doi.org/10.4102/sajim.v10i4.335

Taffel, S. (2016). Perspectives on the postdigital: Beyond rhetorics of progress and novelty. Convergence, 22(3), 324-338. https://doi.org/10.1177/1354856514567827

Vespestad, M. K., & Smørvik, K. K. (2020). Co-Creation as a Tool to Overcome Cross-Cultural Differences in Educational Experiences? Journal of Hospitality & Tourism Education, 32(3), 156–166. https://doi.org/10.1080/10963758.2019.1685391

Villatoro, S., & de Benito, B. (2021). An Approach to Co-Design and Self-Regulated Learning in Technological Environments. Systematic Review. Journal of New Approaches in Educational Research, 10(2), Article 2. https://doi.org/10.7821/naer.2021.7.646

Westheimer, J., & Kahne, J. (2004). What Kind of Citizen? The Politics of Educating for Democracy. American Educational Research Journal, 41(2), 237-269. https://doi.org/10.3102/00028312041002237

Whitton, N. (2018). Playful learning: Tools, techniques, and tactics. Research in Learning Technology, 26(0). https://doi.org/10.25304/rlt.v26.2035

Wolff, A., Gooch, D., Cavero Montaner, J. J., Rashid, U., & Kortuem, G. (2016). Creating an Understanding of Data Literacy for a Data-driven Society. The Journal of Community Informatics, 12(3), 9–26. https://doi.org/10.15353/joci.v12i3.3275

Yates, S. J., Carmi, E., Lockley, E., Wessels, B., & Pawluczuk, A. (2021). Me and my Big Data: Understanding Citizens Data Literacies Research Report (p. 108) [Research Report]. University of Liverpool. https://www.liverpool.ac.uk/media/livacuk/humanitiesampsocialsciences/meandmybiddata/Understanding,Citizens,Data,Literacies,Research,,Report,Final.pdf

Zuboff, S. (2019). The Age of Surveillance Capitalism. The Fight for a Human Future at the New Frontier of Power. PublicAffairs.

Notes