Monográfico

Validation of the OCPBL model for online collaborative project-based learning

Validation of the OCPBL model for online collaborative project-based learning

Montse Guitert Catasús
Universitat Oberta de Catalunya, UOC, España
Teresa Romeu Fontanillas
Universitat Oberta de Catalunya, UOC, España
Marc Romero Carbonell
Universitat Oberta de Catalunya, UOC, España
Pablo Baztán Quemada
Universitat Oberta de Catalunya, UOC, España

Validation of the OCPBL model for online collaborative project-based learning

RIED-Revista Iberoamericana de Educación a Distancia, vol. 27, núm. 2, 2024

Asociación Iberoamericana de Educación Superior a Distancia

How to cite: Romero Carbonell, M., Romeu Fontanillas, T., Guitert Catasús, M., & Baztán Quemada, P. (2024). Validation of the OCPBL model for online collaborative project-based learning. [Validación del modelo ABPCL para el aprendizaje basado en proyectos colaborativos en línea]. RIED-Revista Iberoamericana de Educación a Distancia, 27(2), pp. 159-181. https://doi.org/10.5944/ried.27.2.39120

Abstract: This contribution presents the validation of the Online Collaborative Project-based learning model based on the experience of a subject that is present in all the undergraduate programs in the Universitat Oberta de Catalunya called “ICT Competences”. Based on mixed method research through an online survey that combines quantitative and qualitative data, the opinions of 978 students from the different university programs were gathered. The main aim of this research was to know students’ opinions on the principal elements of the model and its transferability in academic and professional contexts. The contributions of the students are analyzed globally, by gender, and by undergraduate program. The presented results validate the model, but we suggest some adjustments in order to improve some aspects of the implementation process and open further research possibilities based on its application. As the main conclusions of the study, the results suggest the need to sequence students’ asynchronous communication, adapting the different roles of the teams and to provide a more professional-oriented context for some of the activities in order to make the model more closely related to the professional field. As an aspect for improvement, students mentioned some difficulty in managing situations in which the established agreements were not complied with. Among the positive aspects are the relationship with peers, learning new digital tools and resources, acquiring soft skills such as empathy and teamwork, and learning to collaborate effectively online.

Keywords: online learning, collaborative learning, project-based learning, higher education.

Resumen: Se presenta la validación del modelo Aprendizaje Basado en Proyectos Colaborativos en Línea en el marco de una asignatura transversal de la Universitat Oberta de Catalunya denominada “Competencias TIC”. La metodología seguida en esta investigación se enmarca en un diseño de métodos mixtos, combinando datos cualitativos y cuantitativos. Mediante un cuestionario con preguntas cerradas y abiertas se recabó la opinión de 978 estudiantes de los diferentes estudios de esta universidad, donde valoran los elementos fundamentales del modelo, así como su transferibilidad a los ámbitos académico y profesional. Las aportaciones se analizan en global, por género y por estudio. Se valida el modelo, y se sugieren algunas mejoras en aspectos de su implementación, abriendo nuevas posibilidades de investigación: se plantea secuenciar el trabajo asíncrono para permitir su implementación por todo el alumnado, adaptar los roles de los equipos de trabajo y contextualizar algunas actividades al ámbito profesional de cada estudio. Se concluye que el modelo es completamente transferible a otras asignaturas o estudios en línea. Como aspecto mejorable se menciona la dificultad de gestionar las situaciones en las que algún miembro del equipo no cumpla con los acuerdos establecidos, mientras que entre los aspectos positivos destacan la relación con los/las compañeros/as, el aprendizaje de nuevas herramientas y recursos digitales, la adquisición de habilidades relacionadas como la empatía y el aprendizaje de trabajar colaborativamente en línea.

Palabras clave: aprendizaje en línea, aprendizaje colaborativo, aprendizaje basado en proyectos, educación superior.

INTRODUCTION

One of the main methodological innovations of the “Bologna process” in higher education has been placing the student in the centre, a consequence of the incorporation of competencies that connect students with a professional environment ( Montero, 2010 ). This process remains open in response to changes in society, where the need arises to acquire new skills of use for professional development (Díaz-García et al., 2023).

Within the framework of the European Higher Education Area (EHEA), there is a need to incorporate active methodological approaches based on the implementation of the competences to be acquired. Among these active methodologies, collaborative learning has proven to promote competency learning effectively (Hernández et al., 2021; Okolie et al., 2022).

Collaborative learning combines the acquisition of the knowledge required by each discipline with the development of the competences necessary for the world of work, such as problem solving, social and communication skills, and individual collaborative competence, essential for any 21st century professional (Rios et al., 2020).

In the digital era, online work as part of collaborative work is common practice in many organizations, where team members must be able to communicate and collaborate effectively through digital platforms and online tools. This involves skills such as information and data management, organizing time and tasks efficiently, being able to work with people from different backgrounds and even at different times, and the need to remain up to date on the use of online collaborative tools, without neglecting the ethical and civic aspects (Guitert & Romeu, 2020).

To accompany students in this complex process, it is important that teachers know and take into account in their teaching design the differentiating characteristics and implications of online work, which we can find defined in Romero et al (2021), which points out the ten key components for online teaching-learning: active role of students, competencies, active and collaborative methodologies, varied typology of e-activities, synchronous and asynchronous communication, resources for teaching-learning, continuous assessment, role of teachers as guide, planning, stable learning environment, and well-defined tools.

Collaborative learning is usually combined with active methodologies for group activities, one of the most common being Project Based Learning (PBL), characterized by encouraging autonomy, constructive research, achievement of objectives, collaboration, communication, and reflection based on projects anchored in the real world (Kokotsaki et al., 2016). Its growing implementation in various university degrees in different contexts is confirmed (Guo et al., 2020), and increasingly common in online contexts (Hernández-Arvizu et al., 2023).

The complex nature of the teaching-learning processes involved requires establishing systematic models so that their application allows activation of the mentioned aspects (asynchronous communication, continuous assessment, acquisition of competences, etc.) to allow a correct evaluation that involves feedback and improvement of the implemented processes.

Since its adaptation to the EHEA in 2009, the Universitat Oberta de Catalunya has included a transversal subject called “ICT Competencies”, which served as the basis for the design of the Online Collaborative Project-Based Learning (OCPBL) model explained in Guitert et al. (2020), and whose implementation is the focus of this article.

The OCPBL sequences the process into four phases: Starting, Structuring, Development, and Closing and Dissemination, and three transversal axes: open resources, continuous and heterogeneous assessment, and teaching presence. Figure 1 shows the design of the model and its different elements:

Figure 1
ABPCL Model (Guitert et al., 2020)

ABPCL Model (Guitert et al., 2020)

The model can be adapted to different contexts, for example placing greater emphasis on some of the phases based on objectives in a subject. This article shows the validation of the model OCPBL based on the results obtained from a mixed methodological approach during the end of the first semester of the 2022-2023 academic year, where the students’ perceptions of the ICT Competences subject offer a broader understanding of their vision of the application of the model for online learning teamwork. It also explores students' perception of the possibilities of transferring the model to the professional field. Therefore, to focus our research, we ask the following research questions:

METHODOLOGY

This research uses mixed methods combining qualitative and quantitative data (Creswell, 2021) sequentially throughout the research process, both in the collection of information and in its analysis.

Data were gathered through an online questionnaire, appropriate for research in the social field (Babbie, 2017), and for collecting data from a population which is too broad to make a direct observation.

It was based on quantitative questions with answers on a Likert scale from 1 to 5 and a series of open questions based on the model, in which students can express their opinions on the different aspects of the subject.

The questionnaire was validated by the researchers of the Edul@b research group. Subsequently, the internal consistency of the closed questions was reviewed using Cronbach's alpha coefficient (Amirrudin et al., 2021), obtaining an index of 0.94. The redundancy found could be avoided by suppressing the ACAD variables (Table 1), obtaining an index of 0.89, but it was decided to keep them due to the relevance of the insight they provide on the transferability of the model.

The questionnaire asks demographic questions (age, gender, and study area) as well as questions related to the importance of aspects of network teamwork, specified in Table 1.

Table 1
Variables grouped by dimensions

Assess the importance of the following organizational aspects ORG1 [The definition of group agreements]
ORG2 [Review of agreements]
ORG3 [Digital information management (shared files, etc.)]
ORG4 [The initial planning]
ORG5 [Planning review (replanning)]
Actions to communicate synchronously SINC1 [Start an activity]
SINC2 [Finish an activity]
SINC3 [Streamline decision making]
SINC4 [Resolve a conflict]
SINC5 [Social interaction]
Relevance of the following tasks when evaluating your team's work EVAL1 [Self-assessment]
EVAL2 [Co-assessment]
EVAL3 [Assessment of a project by colleagues]
Imagine that you find that one of the members of your group has not made any contributions. Assess the appropriateness of these actions to resolve the situation CONF1 [Contact the person to find out what the problem is]
CONF2 [Talk to the teachers and have them take the appropriate measures]
CONF3 [Request an extension of the deadline while it is resolved]
CONF4 [Continue the work with the rest of the team and say nothing]
CONF5 [Exclude the person from the group and distribute their tasks]
Assess the degree of academic usefulness of the following elements ACAD1 [Group operating agreements]
ACAD2 [Group planning]
ACAD3 [Role distribution]
ACAD4 [Organization of digital information]
ACAD5 [Presentation of digital information]
ACAD6 [Using asynchronous communication]
ACAD7 [Use of synchronous communication]
ACAD8 [Assessment/Reflection of collaboration]
ACAD9 [Conflict resolution strategies]
ACAD10 [Consensus, argumentation, and negotiation]
ACAD11 [Assessment of a project by colleagues]
Assess whether the following items are or can be transferable to your professional environment PROF1 [Group operating agreements]
PROF2 [Group planning]
PROF3 [Role distribution]
PROF4 [Organization of digital information]
PROF5 [Presentation of digital information]
PROF6 [Using asynchronous communication]
PROF7 [Use of synchronous communication]
PROF8 [Assessment/Reflection of collaboration]
PROF9 [Conflict resolution strategies]
PROF10 [Consensus, argumentation, and negotiation]
PROF11 [Assessment of a project by colleagues]

The population comprises the entire student body of the ICT Competence subject in all UOC degree programs where OCPBL is applied, with a total of 3,731 students enrolled at the time of the study (end of the first semester of the 2022-2023 academic year). The administration of the questionnaire was confidential and anonymous, with a Google Form used to deliver it and to monitor the number of responses. The obtained sample is the result of the voluntary participation of students who responded, being representative of each of the courses where the ICTC subject is taught, including 26% of the population with a confidence level of 95% and a margin of error of 2.7%. Table 2 presents the population and sample data of the entire set, as well as the breakdown by study area.

Table 2
Composition of the population and the sample by study area

E1 E2 E3 E4 E5
Law and Political Science Economy and Business Arts and Humanities Psychology and Educational Sciences Informatics and Communication
Population 918 1161 305 654 693
Sample 253 261 120 133 204
% population 28 % 22 % 39 % 20 % 29 %

The data analysis was conducted through a statistical analysis of the quantitative data with the free software program JASP and an analysis of the written discourse (Krippendorff, 2019) with live coding using the free software program QCAmap.

RESULTS AND DISCUSSION

Table 3 shows the results of the mean ratings (Mean), both overall and by gender, and standard deviation (SD) for each variable. The columns “Differences by gender” show the values of the . statistic and the p-value of the Student t test for independent samples. In the cases with significant Brown-Forsythe values, the Welch t test was also applied, obtaining the same conclusions. The columns “Differences by study area” show the F or H statistic for independent samples of more than two groups (an ANOVA or post hoc Dunn test was applied depending on the homogeneity of variances) and the corresponding p or p Holm value in each case.

Table 3
Assessment of teamwork variables: overall, by gender, and by study area

Global Women Men Differences by gender Differences by study
Mean SD Mean SD Mean SD t p F/H* p/pholm*
ORG1 4,17 1,00 4,24 0,97 4,05 1,05 2.910 0.004 1,403 0,231
ORG2 3,96 1,02 4,03 1,01 3,84 1,02 2.879 0.004 2,334 0,054
ORG3 4,38 0,82 4,45 0,80 4,26 0,84 3.498 < .001 0,506 0,731
ORG4 4,20 1,01 4,29 0,95 4,06 1,10 3.496 < .001 3,453 0,008
ORG5 4,24 0,93 4,30 0,93 4,13 0,93 2.730 0.006 3,259 0,011
SINC1 3,73 1,42 3,75 1,44 3,68 1,39 0.758 0.449 22,031* <0,001*
SINC2 3,87 1,33 3,94 1,34 3,77 1,32 1.920 0.055 21,423* <0,001*
SINC3 4,00 1,24 4,02 1,24 3,97 1,23 0.625 0.532 1,952 0,100
SINC4 3,72 1,42 3,76 1,43 3,63 1,41 1.413 0.158 11,205* 0,024*
SINC5 3,45 1,48 3,52 1,49 3,33 1,47 1.909 0.057 21,673* <0,001*
EVAL1 4,11 0,99 4,21 0,96 3,95 1,02 3.856 < .001 2,427 0,046
EVAL2 4,18 0,94 4,25 0,93 4,04 0,96 3.367 < .001 1,913 0,106
EVAL3 4,18 0,97 4,23 0,96 4,01 1,01 2.257 0.024 0,833 0,505
CONF1 4,58 0,82 4,62 0,78 4,50 0,89 2.250 0.025 0,558 0,693
CONF2 3,34 1,33 3,37 1,36 3,30 1,29 0.839 0.402 2,133 0,075
CONF3 2,93 1,37 2,97 1,40 2,86 1,32 1.266 0.206 5,194 <0,001
CONF4 2,78 1,48 2,73 1,49 2,88 1,45 -1.574 0.116 1,592 0,174
CONF5 2,09 1,31 2,00 1,31 2,23 1,30 -2.672 0.008 22,200* <0,001*
ACAD1 4,07 1,11 4,16 1,09 3,92 1,14 3.330 < .001 1,673 0,154
ACAD2 4,28 1,03 4,34 1,01 4,19 1,05 2.154 0.032 1,036 0,388
ACAD3 3,86 1,18 3,90 1,17 3,78 1,19 1.592 0.112 0,557 0,694
ACAD4 4,31 0,91 4,38 0,86 4,17 0,98 3.539 < .001 0,880 0,475
ACAD5 4,34 0,90 4,42 0,85 4,21 0,96 3.557 < .001 1,436 0,220
ACAD6 4,04 1,12 4,09 1,10 3,94 1,15 1.963 0.050 0,720 0,579
ACAD7 4,08 1,15 4,17 1,09 3,92 1,23 3.170 0.002 2,432 0,046
ACAD8 4,22 0,98 4,33 0,93 4,03 1,04 4.707 < .001 2,016 0,090
ACAD9 4,15 1,04 4,28 0,98 3,93 1,11 5.078 < .001 1,536 0,190
ACAD10 4,34 0,96 4,43 0,88 4,20 1,04 3.618 < .001 0,434 0,784
ACAD11 4,18 1,04 4,22 1,01 4,10 1,08 1.710 0.088 0,968 0,424
PROF1 4,12 1,11 4,19 1,07 4,01 1,16 2.455 0.014 2,262 0,061
PROF2 4,34 1,00 4,39 0,98 4,25 1,03 2.107 0.035 10,257* 0,036*
PROF3 3,97 1,18 4,02 1,16 3,87 1,20 1.900 0.058 0,975 0,420
PROF4 4,20 1,01 4,29 0,98 4,06 1,04 3.379 < .001 0,486 0,746
PROF5 4,20 1,02 4,30 0,99 4,04 1,05 3.914 < .001 0,837 0,501
PROF6 4,09 1,19 4,19 1,14 3,91 1,24 3.472 < .001 1,829 0,121
PROF7 4,34 1,01 4,40 0,97 4,22 1,07 2.769 0.006 0,949 0,435
PROF8 4,17 1,04 4,26 1,01 4,02 1,07 3.500 < .001 4,183 0,002
PROF9 4,34 0,96 4,44 0,91 4,17 1,02 4.198 < .001 12,367* 0,015*
PROF10 4,40 0,93 4,45 0,89 4,30 0,99 2.414 0.016 0,780 0,538
PROF11 3,89 1,23 3,97 1,22 3,77 1,25 2.427 0.015 3,534 0,007

Next, the overall results included in the previous table are discussed, along with additional findings from quantitative and qualitative elements to enable a more focused discussion by dimensions and study areas.

The results in Table 3 show that the general rating is very positive, with 28 of the 40 variables scoring an average value greater than 4, nine of them with an average rating between 3 and 4, and three with an average rating between 2 and 3. These results allow for the validation of the model and its overall application, while also helping to focus on areas for improvement.

It should be noted that in 28 of the 40 variables studied the differences between gender are significant and that in only two of the 40 variables the average rating of men is higher than that of women (CONF4 and CONF5).

The open question “After having worked as an online team in the subject, what do you take away as positive and what as negative?”, was answered by 723 students, with the most notable positive aspects being the relationship with classmates, learning new tools and resources, acquiring soft skills such as empathy or responsibility, teamwork, and learning to work collaboratively online. A notable aspect needing improvement was the difficulty of managing some situations in which a member of the team did not comply with the established agreements.

The results regarding the first research question are now discussed:

How do the students of the subject “ICT Competences” value the collaborative methodology within the OCPBL framework?

The results reflected in Table 3 for each of the dimensions of the OCPBL model and its implementation in the ICT Competences (ICTC) subject are discussed, with a cross-sectional approach to the gender variable, and considering the four critical processes of networked collaboration: organization and planning, communication, evaluation, and making collaboration conscious (Guitert, 2022).

Organization

For each of the five ORG variables the following question is answered: “Rate the importance of the following organizational aspects”. Two variables of transfer to the academic and professional field (ACAD3 and PROF3) are also included, with aspects closely related to the organization of teams, such as the distribution of roles.

Initial planning, as well as reviewing and adapting as the project progresses, are key elements for both PBL and online teaching which the model explicitly includes in the Starting, Structuring and Development stages (Figure 1). The students’ high scores for the variables ORG4 (Mean = 4.20, SD = 1.01) and ORG5 (Mean = 4.24, SD = 0.93) reaffirm the usefulness of planning teamwork in online activities.

The definition and review of agreements (ORG1 and ORG2) are also elements that improve the effectiveness of teamwork, and aspects such as responsibility and respect for others' ideas (Pupik et al, 2023). The activities based on the OCPBL model insist on the importance of establishing prior agreements and reviewing them in each of the subsequent phases of the project in order to function more effectively and avoid critical situations or solve them if they occur. Students are satisfied with completing these tasks and find them useful for the development of teamwork: “Although it was challenging at first, creating the operational agreements helped us organize ourselves and prevented issues. We had to redo some aspects a couple of times, but they were very helpful.” (CT68).

The ORG3 variable, digital information management (Mean = 4.38, SD = 0.82), is highly valued by students, and is also one of the aspects highlighted as positive in their qualitative assessment: “Learning in a virtual group environment, use of digital tools within the field of ICT and also in tasks such as creation of digital projects, planning, organization, innovation of ideas, etc.” (CT57).

The assessment of this variable is explained by the sequenced and organized instructions provided in the ICTC subject to help manage the teams' information using Google tools, establishing the bases for the management of digital information throughout their studies.

The ORG variables are scored significantly higher by women than by men, although with small Cohen effect sizes (min = 0.15, max = 0.23).

On the other hand, the variables ACAD 3 and PROF3 with average ratings below 4 (3.86 and 3.97 respectively) lead us to rethink the management of roles in teamwork. The CTIC subject proposes establishing the roles of Coordinator, Secretary, and Manager. To improve this aspect, roles should be adapted to the most common and meaningful tasks of the students’ degree course, following the indications of Sundlin et al. (2022), which provides guidelines on the importance of the context and a clear definition of roles for optimal implementation and in publications such as Belbin and Brown (2022), addressing the roles of professional teams in virtual environments.

Communication

To offer greater geographical and time flexibility to the members of the work teams, the OCPBL model proposes asynchronous communication, which allows studies to be compatible with other obligations. However, they are given the possibility of communicating synchronously at specific times, for example to reach consensus on urgent decisions, in line with other research indicating a good acceptance of synchrony as a supplementary mode of communication (Besser, 2023).

When asking about positive aspects and those needing improvement in the application of the model, we found comments in favour of asynchrony and its discovery: “I have learned a lot about the functioning of the tools used to do the project and the experience of having done work with asynchronous communication” (CT141). However, some difficulties in its application are also expressed: “working asynchronously represents an overload of work if the frequency and work of each person is not planned, since it involves being aware of the response of others” (CT248), although these are unrepresentative cases when compared to the overall responses to the questionnaire. In some cases, the use of synchrony is also valued positively at certain specific moments: “making a record of each meeting we held on Google Meet or a summary of what we talked about on WhatsApp, so that it would be reflected in the forum, taking into account that to carry out a project like this, with so many activities, you have to communicate a lot” (CS347).

To find out in which situations students consider synchronous communication most useful, they were asked to evaluate five actions to be communicated synchronously (Table 3, SINC variables).

There are no significant differences regarding gender, although women tend to have a slightly higher average rating.

Regarding the global assessment, the highest-scoring action on average is the one corresponding to the SINC3 variable: Streamline decision making. Beyond the positive average rating of all of them, we see that in four of them the average rating is less than 4, and in one it is exactly 4, which places the group among the group of 12 out of 40 variables least valued. A more detailed analysis of the scores for each variable (Figure 2) shows the disparity of the ratings, as suggested by the high standard deviation of all of them.

Figure 2
Relative frequencies of the evaluations for each variable SINC in %

Relative frequencies of the evaluations for each variable SINC in %

These results reinforce the priority use of asynchrony as a form of communication in the OCPBL model, while showing that the use of synchrony in some cases is valued positively by a significant percentage of the students (55.10% in the case of less valued variables such as SINC5), so it is reasonable to also maintain it as a reinforcement of asynchronous communication.

Assessment

Another dimension of online collaboration is assessment, which in the OCPBL model proposal is continuous and based on a 360º vision, involving both teachers and students at different times and ways of evaluating (Romeu et al., 2016). To gather the students' perception of the assessment system, they were asked about the relevance of the three assessment methods in the CTIC subject (Table 3, EVAL variables).

The three variables obtain average ratings above 4, with the ratings of those tasks in which the work of others is evaluated (Mean = 4.18) being somewhat higher compared to self-assessment (Mean = 4.11).

The average ratings of the three assessment modes are very similar (4.11, 4.18 and 4.18). This is shown in more detail in Figure 3:

Figure 3
Relative frequencies of the evaluations for each variable EVAL in %

Relative frequencies of the evaluations for each variable EVAL in %

These results confirm “the positive perception of the student body regarding their active role in the assessment of their own learning process” (Cabrera et al., 2023) as progress is made towards the teaching objectives, given that these practices “promote participation and motivation with activities aimed at acquiring the competence of working as a team” (Planas-Lladó et al., 2020).

There are significant differences in the three variables according to gender, with effect sizes of 0.26, 0.22, and 0.15, always with a higher average rating for women. Although the results of the study do not provide a direct explanation for these differences, there is some agreement with studies such as González-Betancor et al. (2019), confirming that women engage in more detailed and reflective self-assessment processes than men.

Critical situations inherent to online collaboration

The implementation of the OCPBL model considers the possibility of conflicts appearing within some work teams, which may be more frequent in teams made up of members with prior knowledge and experience (Edmonson & Harvey, 2018) as is generally the case in the profile of UOC students.

As proposed in O'Neil and McLarnon (2018), dealing with conflicts in a controlled way can be beneficial for a team, since it promotes discussion, sharing different points of view, and the analysis of alternative paths of action. For this reason, an activity is proposed at the beginning so that students become aware of conflicts in teamwork and how to resolve them, participating in a virtual debate about a critical real case, including questions for reflection and action. Furthermore, they are also asked to explain the ways to resolve conflicts, for example, if members do not fulfil their assigned responsibilities. Even so, problems can always arise in the management of collaborative work throughout the process, due to multiple factors, such as a lack of agreement in the group formation process, the difficulties of evaluating individual participation in collective work or, to a greater extent, due to the “free rider” behaviour of some team members ( Ramdeo et al., 2022).

In the open response to the question “After having worked as an online team in the subject, what do you take away as positive and negative?”, the most significant aspect mentioned for improvement is the difficulties in some cases in managing a lack of involvement by other team members: “it irritates me that my colleagues are not as involved as I am in the project” (CS34) is an illustrative example of this discomfort.

Regarding this same aspect, the following closed question was included: “Imagine that you find that one of the members of your group has not made any contribution. Rate the appropriateness of these actions from 1 to 5 to resolve the situation”. The actions for which assessment was requested, as well as the results obtained for each of them, can be seen in variables CONF in Tables 1 and 3 respectively.

It should be noted that the two actions with the lowest rating, CONF4 (mean: 2.78; SD: 1.48) and CONF5 (mean: 2.09; Sd: 1.31), are the ones furthest from what is considered good conflict management (Pazos et al., 2022). The three best valued, and also closest to good conflict management, are enhanced by the characteristics of the OCPBL model, which has continuous assessment and teaching presence as its fundamental axes.

Figure 4 shows the relative frequency of the five levels of assessment for each of these variables:

Figure 4
Relative frequencies of the evaluations for each variable CONF in %

Relative frequencies of the evaluations for each variable CONF in %

We can see how the sum of frequencies of the highest ratings are directly proportional to how appropriate each action is in relation to conflict management (90.74% in the variable CONF1 compared to 17. 59% in CONF5), while in the other three variables the percentage of each of the actions is more balanced, within the trend of each one.

This result reinforces the strategy followed of initially providing a series of resources and guidelines, explicitly addressing conflict management, and sending explicit messages in this regard throughout the course (Ou & Joyner, 2023).

The qualitative data show how teamwork and learning to work collaboratively online are the two most outstanding positive aspects: “It is positive to continue growing in the complex situation of working as a team, also in virtual environments” (CS39).

The results related to the second research question are discussed next.

Are there significant differences between the elements of the model in terms of their transfer in the academic and professional spheres?

The CTIC subject helps students in the different degree courses of the UOC to acquire the digital competences necessary for good performance, both academic and professional. So, in its design and implementation, it promotes the acquisition and transfer of these competencies to other environments, with particular relevance given to teamwork in online networks.

Next, the results of the ACAD and PROF variable groups are analysed (Table 3), which inquire about the academic and professional utility of the elements of the OCBPL model in their implementation in the CTIC subject.

Two variables associated with the management of digital information (ACAD5 and ACAD4) and two variables related to the management of collaborative work (ACAD10 and ACAD2) stand out with positive ratings. “Distribution of roles” stands out as the only one with an average rating of less than 4 (Mean = 3.86; SD = 1.18). Just as the first four reinforce the academic usefulness of online collaborative work and information management, the last variable leads us to rethink how the management of roles in teamwork is considered. The proposed distribution in the CTIC subject has generally established roles, as we already saw when discussing the results of the Organizational Aspects. The proposal for the roles of Coordinator, Secretary or Manager can be reviewed to see if an adaptation of the roles to the students’ degree significantly increases the rating of this item.

In this case, the three highest-scoring variables are related to the management of group work and synchronous communication, while Distribution of roles is also valued below 4, but in this case the variable PROF11 is the one with the lowest average rating (Mean = 3.89; SD = 1.23).

The high ratings of all the variables in both dimensions allow us to affirm the possibilities of both academic and professional transfer of the model and its application. To go deeper into this analysis, Table 4 shows the level of significance of the differences in mean ratings of the same pair of variables for the academic and professional fields.

Table 4
Level of significance of the differences in mean ratings of the same pair of variables for the academic and professional field

t p t p
ACAD1/PROF1 -1627 0.104 ACAD7/PROF7 -7334 < .001
ACAD2/PROF2 -1819 0.069 ACAD8/PROF8 1522 0.128
ACAD3/PROF3 -3169 0.002 ACAD9/PROF9 -6324 < .001
ACAD4/PROF4 3437 < .001 ACAD10/PROF10 -1996 0.046
ACAD5/PROF5 4929 < .001 ACAD11/PROF11 8348 < .001
ACAD6/PROF6 -1452 0.147

Significant differences are observed in the pairs of variables 3 (already discussed), 4, 5, 7, 9 and 11, barely significant in the pair of variables 10, and it is observed that there are no significant differences in the pairs 1, 2, 6 and 8.

The significantly higher rating of the ACAD11 variable (Mean = 4.07) “Assessment of a project by colleagues” compared to that of the PROF11 variable (Mean = 3.89) is justified, since the assessment of the projects is done for an academic purpose and planned specifically for the subject, so there is not much projection beyond this.

The ratings of pairs of variables 4 and 5, referring to the organization and presentation of digital information, are significantly higher in the academic dimension (Mean = 4.31 and Mean = 4.34) than in the professional dimension (Mean = 4.20 and Mean = 4.20), which can be explained by the fact that students put these skills into play in a very concrete way for the execution of the subject's activities. This makes us consider the possibility of contextualizing the activities more deeply by referring to real applications in a simulated work environment.

Synchronous communication (variable 7) has a higher rating in the professional field (Mean = 4.34) than the academic field (Mean = 4.08), probably because presence and/or constant online relationships are associated with work environments, in which asynchrony has less implementation.

Also, strategies for conflict resolution (variable 9) have a significantly higher value in the professional field (Mean = 4.34) than in the academic field (Mean = 4.15). This fact could be explained because conflicts are not very frequent in the subject, given the resources discussed when talking about the critical situations that students may find useful in future situations of their own professional environment.

To evaluate in more detail the transfer of the model and its implementation in the academic field, we now analyse the results obtained from comparing the average ratings of the variables across different study areas (Table 3).

Figure 5 shows the average ratings of each group of variables per study area.

Figure 5
Average ratings of each group of variables per study area

Average ratings of each group of variables per study area

The variables EVAL, ACAD, and PROF show very similar average ratings across all study areas, with lower ratings in E3 and higher ratings in E4. In the SINC group, the averages are lower in each case compared to the other variables, with study areas E2 and E4 having higher ratings than the other three. In the ORG variable group, study area E3 has a lower average than the rest, which are very similar to each other.

This fact can be explained by the student profile, given that, on the one hand, Arts and Humanities students (E3) are more likely to be studying for personal fulfilment, which means these aspects have less value for their professional development, while E2 and E4 students usually study a degree which will be put to use in the professional field, and in turn, are more aware of the importance of the psychological processes inherent to teamwork.

The group of variables SINC shows lower means compared to the rest of the variables in all study areas. This result is consistent with the emphasis placed on asynchrony from the CTIC subject, as discussed earlier in the Communication section.

Considering the level of significance of these differences (Table 3), no significant differences have been found in the assessment of the group of ACAD variables, while the SINC and PROF variable groups are the ones that present a greater number of variables with significant differences and effect sizes higher than the rest.

Analysing the group of ORG variables in more detail, two of them show significant differences among the study areas: ORG4 with p = 0.008 and ORG5 with p = 0.011. Figure 6 shows between which study areas these differences occur, as well as the effect size in each case.

Figure 6
ORG variables with significant differences between study areas

ORG variables with significant differences between study areas

We see that in these two variables, students from Arts and Humanities (E3) give average ratings below those of other study areas for the initial planning (ORG4) compared to E1, E2, and E4, and for the review of planning (ORG5) compared to E4 and E5.

The results allow us to confirm these differences, as well as those related to the rest of the variables, but they do not allow us to delve into the reasons for them, a matter that is beyond the scope of the present study. However, they provide sufficient evidence to generate instruments that help understand the reasons for the differences found.

CONCLUSIONS

Answering the first research question, the results show how positively students value the teamwork methodology of the OCBPL model, validating the model and its application as a whole, while also highlighting areas for improvement.

Aspects related to teamwork evaluation and organization are highly valued, suggesting improvement by assigning roles for teamwork tailored to the professional profiles of different study areas.

Given the difficulties that some students identified with asynchronous communication inherent in the model, a better sequencing of this type of communication is proposed as an improvement, providing more guidelines for its implementation and using synchronous communication at specific moments.

Including activities to raise awareness of what is involved in teamwork in a network is noted as important for reducing conflict, with learning how to work this way being one of the most highly valued aspects.

Regarding gender differences in evaluation, except for 2 out of 40 variables, the women's average rating is higher, with significant differences observed in 28 of them. Promoting gender equity in team formation, taking into account the reality of student demographics in each case, could harness the potential that these differences bring.

Regarding the second research question, the results support the complete transferability of the model to other subjects within the university or at other online universities, as well as to the professional sphere. While ratings are very positive, differences between some study areas have been found, inviting to further adaptation of activities to the professional reality of different fields of study.

As a limitation of the study, it should be noted that the model was designed and applied only in a specific subject, although this is mandatory and has a high impact in terms of the number of students, which plausibly restricts its transferability to other environments and subjects. Adapting and applying it to other university subjects and to other universities conducting fully-online training actions would help to further define and validate a globally transferable model.

This research has allowed us to know first-hand students' evaluation of the model, validating it globally. On the other hand, it also opens up future lines of research such as exploring the underlying reasons for gender and study-area differences.

REFERENCES

Amirrudin, M., Nasution, K., & Supahar, S. (2021). Effect of Variability on Cronbach Alpha Reliability in Research Practice. Jurnal Matematika, Statistika dan Komputasi, 17(2), 223-230. https://doi.org/10.20956/jmsk.v17i2.11655

Babbie, E. R. (2017). The basics of social research (Seventh edition). Cengage Learning.

Belbin, R. M., & Brown, V. (2022). Team Roles at Work (3.a ed.). Routledge. https://doi.org/10.4324/9781003163152

Besser, E. D. (2023). Making an Impact in Online Learning: Google Chat as a Mechanism to Facilitate the COI Framework. Technology, Knowledge and Learning, 29(1), 413-432. https://doi.org/10.1007/s10758-023-09673-7

Cabrera, N., Fernández-Ferrer, M., Maina, M. F., & Sangrà, A. (2023). Diseño de una propuesta de autoevaluación para el desarrollo de la autorregulación en educación superior. RIED-Revista Iberoamericana de Educación a Distancia, 26(1), 223-244. https://doi.org/10.5944/ried.26.1.34028

Creswell, J. W. (2021). A concise introduction to mixed methods research (Second edition). SAGE.

Díaz-García, I., Almerich, G., Suárez-Rodríguez, J., & Orellana, N. (2023). University students’ competences in ICT: A view from the education domain. Australasian Journal of Educational Technology, 133-151. https://doi.org/10.14742/ajet.6820

Edmondson, A. C., & Harvey, J.-F. (2018). Cross-boundary teaming for innovation: Integrating research on teams and knowledge in organizations. Human Resource Management Review, 28(4), 347-360. https://doi.org/10.1016/j.hrmr.2017.03.002

González-Betancor, S. M., Bolívar-Cruz, A., & Verano-Tacoronte, D. (2019). Self-assessment accuracy in higher education: The influence of gender and performance of university students. Active Learning in Higher Education, 20(2), 101-114. https://doi.org/10.1177/1469787417735604

Guitert, M. (2022). Networked collaboration for teachers and for students. In A. Sangrà (Coord.), Improving online teaching: Practical guide for quality online education. UOC. ISBN 978-84-9180-937-1.

Guitert, M., & Romeu, T. (2020). Aprendizaje colaborativo en red en la universidad. In D. Cañabate & J. Colomer (Coords.), El aprendizaje cooperativo en la universidad del siglo XXI. Propuestas, estrategias y reflexiones. (pp. 95-106). Grao.

Guitert, M., Romeu, T., & Romero, M. (2020). Elementos clave para un modelo de aprendizaje basado en proyectos colaborativos online (ABPCL) en la Educación Superior. American Journal of Distance Education, 34(3), 241-253. https://doi.org/10.1080/08923647.2020.1805225

Guo, P., Saab, N., Post, L. S., & Admiraal, W. (2020). A review of project-based learning in higher education: Student outcomes and measures. International Journal of Educational Research, 102, 101586. https://doi.org/10.1016/j.ijer.2020.101586

Hernández, I., Lay, N., Herrera, H., & Rodríguez, M. (2021). Estrategias pedagógicas para el aprendizaje y desarrollo de competencias investigativas en estudiantes universitarios. Revista de Ciencias Sociales. https://doi.org/10.31876/rcs.v27i2.35911

Hernández-Arvizu, B. F., Guzmán-Cedillo, Y. I., & Lima Villeda, D. N. (2023). Aprendizaje basado en proyectos: La perspectiva del alumnado en aprendizaje semipresencial y totalmente en línea. Revista de Estudios e Investigación en Psicología y Educación, 10(1), 95-111. https://doi.org/10.17979/reipe.2023.10.1.9527

Kokotsaki, D., Menzies, V., & Wiggins, A. (2016). Project-based learning: A review of the literature. Improving Schools, 19(3), 267-277. https://doi.org/10.1177/1365480216659733

Krippendorff, K. (2019). Content Analysis: An Introduction to Its Methodology. SAGE Publications, Inc. https://doi.org/10.4135/9781071878781

Montero Curiel, M. L. (2010). El proceso de Bolonia y las nuevas competencias. Tejuelo: Didáctica de la Lengua y la Literatura, 9.

Okolie, U. C., Oluka, B. N., Oluwayemisi, F. B., Achilike, B. A., & Marcel Ezemoyih, C. (2022). Overcoming obstacles to collaborative learning practices: A study of student learning in higher education-based vocational education and training. International Journal of Training Research, 20(1), 73-91. https://doi.org/10.1080/14480220.2021.1965904

O’Neill, T. A., & McLarnon, M. J. W. (2018). Optimizing team conflict dynamics for high performance teamwork. Human Resource Management Review, 28(4), 378-394. https://doi.org/10.1016/j.hrmr.2017.06.002

Ou, C., & Joyner, D. (2023). Seven years of online project-based learning at scale. International Journal on Innovations in Online Education, 7(1), 69-85. https://doi.org/10.1615/IntJInnovOnlineEdu.2023049968

Pazos, P., Pérez-López, M. C., & González-López, M. J. (2022). Examining teamwork competencies and team performance in experiential entrepreneurship education: Emergent intragroup conflict as a learning triggering event. Education + Training, 64(4), 461-475. https://doi.org/10.1108/ET-06-2021-0208

Planas-Lladó, A., Feliu, L., Arbat, G., Pujol, J., Suñol, J. J., Castro, F., & Martí, C. (2020). An analysis of teamwork based on self and peer evaluation in higher education. Assessment & Evaluation in Higher Education, 46(2), 191-207. https://doi.org/10.1080/02602938.2020.1763254

Pupik, C. G., Grossman, P., Enumah, L., Herrmann, Z., & Kavanagh, S. S. (2023). Core practices for project-based learning: Learning from experienced practitioners in the United States. Teaching and Teacher Education, 133, 104275. https://doi.org/10.1016/j.tate.2023.104275

Ramdeo, S., Balwant, P., & Fraser, S. H. (2022). Not another team assignment! Student perceptions towards teamwork at university management programs. Higher Education, Skills and Work-Based Learning, 12(6), 1122-1137. https://doi.org/10.1108/HESWBL-01-2022-0015

Rios, J. A., Ling, G., Pugh, R., Becker, D., & Bacall, A. (2020). Identifying Critical 21st-Century Skills for Workplace Success: A Content Analysis of Job Advertisements. Educational Researcher, 49(2), 80-89. https://doi.org/10.3102/0013189X19890600

Romero, M., Romeu, T., Guitert, M., & Baztán, P. (2021). The Transition To Online Teaching In Times Of Pandemic: An Experience Of Teachers’ Training In European Higher Education. In SCUOLA DEMOCRATICA (Eds.), Book of Abstracts of the International Conference of the journal Scuola Democratica. Reinventing Education (Associazione “Per Scuola Democratica”).

Romeu, T., Romero, M., & Guitert, M. (2016). E-assessment process: Giving a voice to online learners. International Journal of Educational Technology in Higher Education, 13(1), 20. https://doi.org/10.1186/s41239-016-0019-9

Sundlin, A.-L., Söderhjelm, T. M., & Sandahl, C. (2022). Making rapid shifts in work roles – an essential teamwork skill. An exploratory study of facilitating and inhibiting factors. Team Performance Management: An International Journal, 28(7/8), 461-475. https://doi.org/10.1108/TPM-01-2022-0003


Reception: 01 December 2023
Accepted: 06 March 2024
OnlineFirst: 19 April 2024
Publication: 01 July 2024