INTRODUCTION

Since the turn of the century, Higher Education institutions around the world have increasingly invested in educational technology and learning management systems (Bond et al., 2020; Müller & Wulf, 2020) and even more so in the Covid-19 pandemic (Abu Talib et al., 2021; Z. Chen et al., 2021). Asynchronous Online Discussion (AOD) is a didactic method used a little while back via Information and Communication Technologies, which has had an appreciable impact and usefulness (Andresen, 2009; Fehrman & Watson, 2020; Gao et al., 2013; Thomas, 2013).

Online discussion favors the development of complex learning at university (Jeong & Chiu, 2020; Tirado Morueta et al., 2016), increases student-student interaction (Almatrafi & Johri, 2019), helps establish networked learning communities (Saqr et al., 2020), and works for the social construction of knowledge (Al-Dheleai et al., 2020). The advantage of asynchronous debates is to give students time to learn about a discussion topic, build their position about it, and then offer an argued response to their classmates.

Moreover, several studies published in the last years analyzed the social networks that emerged from interaction during the AODs (e.g., Garcia-Garcia et al., 2021; Lee et al., 2021; Zou et al., 2021). The results from social network analysis (SNA) showed that AOD fosters learning how to influence other people through dialogue and deliberation, beyond disciplinary learning in a subject.

Despite the benefits of AOD, controversy has lately arisen in the academic community about the quantity versus quality of student activity on virtual platforms. One of the issues is that the time they spent online may have misled the results of the debates in previous studies (e.g., Campbell et al., 2008; Pulford, 2011). The problem with measuring online time is that it lacks relevant practical information to know the students’ engagement or participation online. Another problem is that, while some research focused on message length to assess learning outcomes (e.g., Brooks & Bippus, 2012; Vázquez-Cano et al., 2015), we doubt whether the message length is associated with academic performance or influence on classmates.

This background led us to design and conduct a study to provide empirical evidence to confirm or rule out the importance of message length and delay in academic performance and student influence in undergraduate AODs.

LITERATURE REVIEW

The time spent online may be misleading the results of AOD

The latest reviews of AOD research (Almatrafi & Johri, 2019) revealed that the assessment of discussions often consists of counting the time spent online (Campbell et al., 2008; Pulford, 2011) or the number of posts (Saadatdoost et al., 2015). The problem is that these factors serve more to record the level of online activity than to assess learning, especially connection time since we do not know the offline work time involved in writing a post. It is more, we cannot even be sure that students stay working the whole time logged into a learning management system like Moodle or Blackboard because we ignore what they might be doing behind the screen.

Although the time online and the number of posts require to manage digital skills, these skills are a condition to participating in AOD (Junus et al., 2019; Onyema et al., 2019) and do not constitute a learning outcome from the discussion. In any case, it would be reasonable to think that those students who post less frequently have invested more time in crafting the content of their messages, but we have no evidence about that. So far, some studies have analyzed the effect of the delay in posting messages to the forum, but it was the teacher’s delay, not the students’ (Mazzolini & Maddison, 2007). That led us to wonder if there is any relationship between the time it takes students to post and their academic performance or other learning outcomes.

We ignore whether message length is associated to academic performance

Research has long taken students’ word count in forums to measure their interaction, engagement, and even performance (e.g., Brooks & Bippus, 2012; Vázquez-Cano et al., 2015). Many authors still consider word count as a relevant sample data and report it (e.g., Hülsmann & Shabalala, 2016; Sanganyado & Nkomo, 2018; Stephens et al., 2019). In fact, a recent study measured the number of words and assumed it was equivalent to students’ social involvement in AODs (Law et al., 2020).

Concerning teamwork, we recognize that sentence length and message word count usually contribute to better team member performance (Ahuja et al., 2020), but that does not necessarily entail gains in the students’ qualifications. It will depend on the assessment criteria and the aims of each subject at the university. Nor does performance on teamwork need to mean that students develop other individual aspects of their learning process at college, such as managing the information they share in discussion forums or the metacognitive skills to schedule their work.

Regardless of grades, another study found a significant difference in the variance of message length when students were informing their classmates and when they were referring to the task situation (Chávez et al., 2016). After all, it seems that certain words could predict academic achievement, especially when they have a relevant and qualitative connection to the discussion topic (Lin et al., 2020; Yoo & Kim, 2012, 2014).

Perhaps this is why some evidence showed that the number of words students write in an AOD predicts academic performance (Abe, 2020). Nonetheless, it would be necessary to conduct more correlation studies to reinforce the empirical evidence and verify that the results were not due to perturbing variables, such as the importance of the words for the topic of discussion and the information shared in the posts.

The contribution of SNA to assess the student influence in AODs

Beyond grades, AOD involves a formative process based on constructivist pedagogy and the socio-cognitive approach. It develops higher mental functions from the student-student interaction (Bandura, 1986; Vygotsky, 1978), facilitating the accommodation of new ideas into prior cognitive structures (Ausubel et al., 1968; Greco & Piaget, 1959).

In this line, students learn when they give validity to the arguments of their peers (Habermass, 1984) and, therefore, the influence they exert on each other is a relevant factor to learn. However, students’ grades and engagement (i.e., time spent online, number of posts, website visits, and similar factors) do not always reflect their influence on classmates. That may depend on the teacher’s assessment criteria and the quality of the messages. Perhaps the length and delay of posts bear some association with student influence, independent of grades.

SNA provides metrics on the students’ interaction patterns, which may be more or less active or isolated in the network (García-Álvarez et al., 2018). In particular, it allows assessing interaction from centrality metrics (da Silva et al., 2019; Lee et al., 2021; Zou et al., 2021), which indicate how much and in what sense each student’s messages are important for their classmates. We have already taken centrality as an indicator of influence and proposed to use this information for formative objectives during a course (Garcia-Garcia et al., 2021).

Centrality indicates a location for each student relative to the network, and some centrality metrics report on the ability to influence the arguments of others. That does not mean that occupying a central position in the network ensures real student influence. That will depend on the content of the messages. A long post with hardly any information will influence without content. In other words, there will be a formal influence, but it will not change others’ minds. Still, a central position makes it easier for students to influence the reasoning of others. Thus, in this study, we included centrality as a learning outcome from AOD, in addition to grades.

Research questions

This study aimed to test the hypothetical correlation of message length and delay in AODs with academic achievement and influence on classmates. We answered the following research questions (RQ).

• RQ1. Was the message word count associated with the students’ grades?
• RQ2. Was the message word count associated with the students’ centrality?
• RQ3. Was the delay in sending a message associated with the students’ grades?
• RQ4. Was the delay in sending a message associated with the students’ centrality?

METHOD

We designed a two-group posttest-only study because the focus was the association of message words and delay in sending with students’ marks and their influence on classmates. The study focused on proving or ruling out the existence of correlations to optimize efforts for predictive designs in AOD research.

Sample and participants

The sample was 1283 messages by 93 students of Educational Sciences who participated in AODs during the first year of their undergraduate studies. The activity endured ten weeks for 48 of them and fourteen weeks for the others. The teachers moderated the debate and participated in the AODs only to initiate the discussion threads. We excluded the teachers’ messages from the analysis.

All students were between 18 and 31 years old (Mean = 19.84, Standard deviation = 2.16). Seventy-seven of the students were female (82.80%), and the rest were male. We admitted the gender imbalance because it corresponded with the population data of our university. According to the institution’s yearbook, more than 85% of the students enrolled in Educational Sciences were female during the last three academic years.

Table 1 shows the features of the sample. We reported median values and interquartile ranges instead of mean and standard deviation because data were not normally distributed, and we considered robust central tendency and dispersion measure more representative.

Learning environment

The teachers set up discussion forums in Moodle with a semi-structured format (Dommett, 2019; Hammond, 2019). The activity in the forums consisted of students following a discussion thread, posting comments on a lesson of the course. The contributions had to refer to other messages that classmates previously posted. The teachers used the Urkund system to detect plagiarism in each post and progressively penalized the students’ marks from plagiarism higher than 15%.

It is probable that providing alerts, reminders, or somehow facilitating the activity in the forum help students read and respond to their peers more frequently (Wang & Yang, 2012). However, these automatic notifications could reduce their cognitive load and negatively affect their attention span (Sachdeva & Gilbert, 2020). Thus, we suggested the self-management of the activity and using the Moodle search engine to find messages of interest instead of providing any additional facilities. In doing so, we aimed to achieve the participation required for the proper development of the AOD (Hew & Cheung, 2008).

We also disabled the option to modify messages in the forum once posted, as we needed to assess the students’ progress since the original posting date. The students had no access to their grades in the AOD nor to the overall qualifications of the course. Thereby this information could not influence their performance during the debate.

Procedure and data management

At the end of the semester, we downloaded the data of the posts directly from the forum in Moodle. With this procedure, we obtained the word count in each message and the grades, but not a score that would provide information on how long it took students to post their comments. Then, we calculated the message delay based on the minutes elapsed over the weeks in the range of 0-168 hours (one week).

We considered each week an isolated block of time, starting every Monday a new period to comment on the discussion topic and including weekends because some students found it easier to participate in the discussion on non-working days. We recommended they post at least once a week to encourage discussion.

The students had to include references to the previous remarks of their peers in the text they were posting every time. That allowed us to obtain an adjacency matrix with all interactions to analyze the network that emerged from the debate. The SNA provided centrality scores for each student that helped to detect candidates with a high capacity to influence the arguments of their classmates.

Data analysis

We first computed the students’ centrality scores as an indicator of influence on their peers. Then, we assigned their marks and their centralities in the discussion networks to each post and conducted the correlation analysis. The following sections contain details on the SNA and the hypothesis testing.

Social Network Analysis

AODs generated social networks that made it possible to assess the students’ connectivity. The simplest way to obtain centrality is to sum the number of links a node 𝑣 ∈ 𝑉 had, with 𝑣 being the node and 𝑉 being the set of nodes in the network. That is known as degree centrality. These links are the degrees that each node had, representing the nodes to students. In a directed network such as the one we analyzed, there were in-degrees when the node was the final vertex of the link or edge and out-degrees when the node was the initial vertex.

Still, we obtained the degree of connectivity of each person and used it to calculate their eigenvector (EV) centrality instead of degree centrality. We dispensed with degree centrality because it reflected only the number of connections when students addressed others –outdegree– or other classmates interpellated them –indegree– (Diestel, 2017). Instead, we took EV centrality as a reference because it corresponds to the principal eigenvector of the network adjacency matrix. Therefore, it provided more information about the influence that some students had on the insights of others during the discussion.

Figure 1 helps to understand the difference between the two concepts. It contains a random network with 200 nodes representing students who participate in an AOD with a .05 probability of connection. In the version on the left, the nodes with a higher degree appear painted in darker blue. The version on the right is the same but based on EV centrality instead of degree centrality.

Students with higher EV centrality interact with many peers who, in turn, are well connected to others within the online learning community (Negre et al., 2018; Newman, 2010). We used EV centrality to detect students who may spread their information successfully during the debate, sharing meaningful learning content with their peers.

We computed EV centrality from the product of Ax, with A being the adjacency matrix and x being the vector that resulted from the degree centralities of those students directly connected to a person (Bonacich, 1972; Sun & Tang, 2011). Thus, EV centrality correlates strongly with degree centrality, although it is somewhat more associated with indegree than with outdegree (He & Meghanathan, 2016; Valente et al., 2008), and considers the level of connection of the people to add or subtract weight to the value of the student-to-student links.

Correlation analysis

We obtained nonparametric correlation coefficients because of the non-compliance with the normality assumption, and in a preliminary exploration, we ruled out the possibility of a linear relationship. Figure 2 provides scatter plots from that exploration with smooth lines from estimates of the conditional mean function and 95% confidence intervals in grey for each pair of variables. Given the findings, we looked for evidence about monotonic relationships to assume or discard message words and the delay in sending as potential predictor variables for qualifications and influence on others during an AOD.

Considering that correlation coefficients would be low, we computed Vovk-Sellke Maximum p-Ratio (Sellke et al., 2001; Vovk, 1993) to report the likelihood of a particular p-value. The p-value gives the probability of obtaining results in a test that are at least as extreme as the results observed in the data, assuming the null hypothesis of no correlation is correct. When the p-value is small, an extreme outcome would be unlikely under the null hypothesis, and we then rule out the absence of correlation.

In other words, the p-value provides information about the probability of the data given a distribution. However, the Maximum p-Ratio provided information about how many times more likely a p-value is to occur under the alternative against the null hypothesis. That allowed us to discard message length and its delay in further studies on AOD with undergraduate students. In this case, the Maximum p-Ratio was based on the p-value, and the maximum possible odds in favor of the alternative hypothesis over null equals 1/(-e p log( p )) for p ≤ .37.

RESULTS

Nonparametric tests revealed weak correlation coefficients. We detected a low association between grades and either the message words or the delay in posting. There appeared to be a slight tendency to achieve more EV centrality as students took more time to post their comments in the forum. We also found a weak trend to achieve more EV centrality when messages were more synthetic. However, we did not obtain values in the coefficients that would allow us to infer more than a low correlation in any of the four hypotheses.

The level of correlation detected for the grades was significant (p < .05), and even more so for the associations with EV centrality (p < .001). After computing the Maximum p-Ratio from the test for grades and message words, we found it was five times more likely for the p-value to occur under the alternative hypothesis against the null.

In the test for the correlation between the marks and the delay in sending a post, the p-value was seven times more likely, and it was much prominent in the tests for EV centrality. That means that there was a low correlation in both tests, and we can affirm it in a significant and likely assertion. Table 2 contains the correlation coefficients, two-tailed significance, and Maximum p-Ratio.

DISCUSSION

For decades, research on AOD sometimes assessed the engagement and outcomes of the students measuring the time they spent online (Campbell et al., 2008; Pulford, 2011) and the length of their messages (Brooks & Bippus, 2012; Law et al., 2020; Vázquez-Cano et al., 2015). Literature review showed that this practice would be controversial because it does not correctly reflect student learning outcomes (Almatrafi & Johri, 2019).

Our study provided empirical evidence pointing to a low association between message length and delay with academic achievement and the influence on classmates. Although the association was statistically significant, it was not of meaningful practical relevance. In the following sections, we suggest evidence-based guidelines for educational practice and discuss the limitations and implications of the study for future research.

Implications for educational practice

The length of time an assignment lasts can affect student engagement in virtual learning environments. A recent study examined online teaching practices during the Covid-19 pandemic and found that the time elapsed between the beginning of the task and the last submission of the students was associated with academic performance (Schmitz & Hanke, 2021).

Indeed, student engagement influences academic achievement and motivation to learn and reduces apathy and dropout rates (Assunção et al., 2020; Maguire et al., 2017). However, our study showed the relationship between time and performance was scarce when we analyzed the delay in posting messages in an undergraduate online discussion.

Although we remain unaware of the students’ work during the time they spend online, the findings showed that those who take longer to publish a post do not necessarily get better grades, nor do they have much more influence on others’ arguments. Consequently, it makes no sense to encourage students to submit their posts earlier or later, believing that it will help them. They should meet deadlines when applicable, but the time they take to send a message will not improve their academic performance or the acceptance from the rest of the class.

Message length is also not an indicator of quality. We found a weak trend to achieve more EV centrality when messages were more synthetic, but it was slight (r_s = -.229, p < .001), and it would not be advisable to encourage students to write longer or shorter comments. Instead, it would be reasonable to teach them to write messages that make sense for the discussion topic and, if possible, that do not leave the discussion thread.

Previous studies reached similar conclusions in online activities (Jivet et al., 2020), particularly in online discussions using network analysis to assess outcomes (Amastini et al., 2020; Lahuerta-Otero et al., 2019). That reinforces the research pointing that only the number of words related to the discussion topic goes so far as to affect student performance (Lin et al., 2020; Yoo & Kim, 2012, 2014).

Limitations and emerging research

Measuring academic success from grades was somewhat limited. Students develop knowledge and skills aside from course objectives or assessment criteria and nevertheless are beneficial for professional development. Qualifications may reflect the adaptive ability of the students to the teachers’ demands rather than the academic performance itself.

EV centrality is also not a thorough indicator of influence. It indicates a position in the network with vast possibilities to influence peers, but it may not include meaningful content to learn about the discussion topic. After all, SNA is increasingly used to analyze the spread of disease (Block et al., 2020; Silk et al., 2017) or organized crime networks (Bouchard, 2020; Burcher & Whelan, 2018), and situations of disrespect and aggression can occur in AODs (Yapici & Akbayin, 2012).

Students with a higher level of EV centrality could propagate this kind of content, thus exerting a negative influence. Teachers’ modeling the forms of interaction (Choi & Johnson, 2005; Smet et al., 2010) and adopting strategies for moderating the debate (N.-S. Chen et al., 2011) should be enough to avoid these situations. However, mitigating negative behavior does not mean that the most influential students always share positive content or at least content that helps their classmates to learn about the discussion topic. Even so, EV centrality and grades provided a more complete and rigorous measure than other alternatives, such as self-report tests.

In this line, we consider it relevant to analyze the content of the messages and not only the structure of the networks, as pointed out by previous studies (Garcia-Garcia et al., 2021; Jan & Vlachopoulos, 2019). The content of the posts would be more effective than message length and delay in helping students to learn and exert a positive influence on others. Therefore, in future research on the impact of AOD on academic performance or student-student communication, word count or the delay in sending a post will be of little interest.

REFERENCES

Abe, J. A. A. (2020). Big five, linguistic styles, and successful online learning. The Internet and Higher Education, 45, 1-9. https://doi.org/10.1016/j.iheduc.2019.100724

Abu Talib, M., Bettayeb, A. M., & Omer, R. I. (2021). Analytical study on the impact of technology in higher education during the age of COVID-19: Systematic literature review. Education and Information Technologies, 26, 6719-6746. https://doi.org/10.1007/s10639-021-10507-1

Ahuja, R., Khan, D., Symonette, D., Pan, S., Stacey, S., & Engel, D. (2020). Towards the Automatic Assessment of Student Teamwork. Companion of the 2020 ACM International Conference on Supporting Group Work, 143-146. https://doi.org/10.1145/3323994.3369894

Al-Dheleai, Y. M., Tasir, Z., & Jumaat, N. F. (2020). Depicting Students’ Social Presence on Social Networking Site in Course-Related Interaction. SAGE Open, 10(1), 1-8. https://doi.org/10.1177/2158244019899094

Almatrafi, O., & Johri, A. (2019). Systematic Review of Discussion Forums in Massive Open Online Courses (MOOCs). IEEE Transactions on Learning Technologies, 12(3), 413-428. https://doi.org/10.1109/TLT.2018.2859304

Amastini, F., Sari Kaunang, C. P., Nefiratika, A., Sensuse, D. I., & Lusa, S. (2020). Collaborative Learning in Virtual Learning Environment using Social Network Analysis: Case study Universitas Terbuka. In Institute of Advanced Engineering and Science (Ed.), 2020 7th International Conference on Electrical Engineering, Computer Sciences and Informatics (EECSI) (pp. 262-269). IEEE. https://doi.org/10.23919/EECSI50503.2020.9251904

Andresen, M. A. (2009). Asynchronous discussion forums: success factors, outcomes, assessments, and limitations. Journal of Educational Technology & Society, 12(1), 249-257. https://www.jstor.org/stable/jeductechsoci.12.1.249

Assunção, H., Lin, S. W., Sit, P. S., Cheung, K. C., Harju-Luukkainen, H., Smith, T., Maloa, B., Álvares Duarte Bonini Campos, J., Ilic, I. S., Esposito, G., Francesca, F. M., & Marôco, J. (2020). University Student Engagement Inventory (USEI): Transcultural Validity Evidence Across Four Continents. Frontiers in Psychology, 10, 1-12. https://doi.org/10.3389/fpsyg.2019.02796

Ausubel, D. P., Novak, J. D., & Hanesian, H. (1968). Educational psychology: a cognitive view. Holt, Rinehart and Winston.

Bandura, A. (1986). Social Foundations of Thought and Action: A Social Cognitive Theory. Prentice-Hall, Inc.

Block, P., Hoffman, M., Raabe, I. J., Dowd, J. B., Rahal, C., Kashyap, R., & Mills, M. C. (2020). Social network-based distancing strategies to flatten the COVID-19 curve in a post-lockdown world. Nature Human Behaviour, 4(6), 588-596. https://doi.org/10.1038/s41562-020-0898-6

Bonacich, P. (1972). Factoring and weighting approaches to status scores and clique identification. The Journal of Mathematical Sociology, 2(1), 113-120. https://doi.org/10.1080/0022250X.1972.9989806

Bond, M., Buntins, K., Bedenlier, S., Zawacki-Richter, O., & Kerres, M. (2020). Mapping research in student engagement and educational technology in higher education: a systematic evidence map. International Journal of Educational Technology in Higher Education, 17, 1-30. https://doi.org/10.1186/s41239-019-0176-8

Bouchard, M. (2020). Collaboration and Boundaries in Organized Crime: A Network Perspective. Crime and Justice, 49, 425-469. https://doi.org/10.1086/708435

Brooks, C. F., & Bippus, A. M. (2012). Underscoring the Social Nature of Classrooms by Examining the Amount of Virtual Talk across Online and Blended College Courses. European Journal of Open, Distance and E-Learning2, 1, 1-8. https://eric.ed.gov/?id=EJ979602

Burcher, M., & Whelan, C. (2018). Social network analysis as a tool for criminal intelligence: understanding its potential from the perspectives of intelligence analysts. Trends in Organized Crime, 21(3), 278-294. https://doi.org/10.1007/s12117-017-9313-8

Campbell, M., Gibson, W., Hall, A., Richards, D., & Callery, P. (2008). Online vs. face-to-face discussion in a web-based research methods course for postgraduate nursing students: A quasi-experimental study. International Journal of Nursing Studies, 45(5), 750-759. https://doi.org/10.1016/j.ijnurstu.2006.12.011

Chávez, J., Montaño, R., & Barrera, R. (2016). Structure and content of messages in an online environment: An approach from participation. Computers in Human Behavior, 54, 560-568. https://doi.org/10.1016/j.chb.2015.08.046

Chen, N.-S., Kinshuk, Wei, C.-W., & Liu, C.-C. (2011). Effects of matching teaching strategy to thinking style on learner’s quality of reflection in an online learning environment. Computers & Education, 56(1), 53-64. https://doi.org/10.1016/j.compedu.2010.08.021

Chen, Z., Jiao, J., & Hu, K. (2021). Formative Assessment as an Online Instruction Intervention. International Journal of Distance Education Technologies, 19(1), 50-65. https://doi.org/10.4018/IJDET.20210101.oa1

Choi, H. J., & Johnson, S. D. (2005). The Effect of Context-Based Video Instruction on Learning and Motivation in Online Courses. American Journal of Distance Education, 19(4), 215-227. https://doi.org/10.1207/s15389286ajde1904_3

da Silva, L. F. C., Barbosa, M. W., & Gomes, R. R. (2019). Measuring Participation in Distance Education Online Discussion Forums Using Social Network Analysis. Journal of the Association for Information Science and Technology, 70(2), 140-150. https://doi.org/10.1002/asi.24080

Diestel, R. (2017). Graph Theory (5th ed.). Springer. https://doi.org/10.1007/978-3-662-53622-3

Dommett, E. J. (2019). Understanding student use of twitter and online forums in higher education. Education and Information Technologies, 24(1), 325-343. https://doi.org/10.1007/s10639-018-9776-5

Fehrman, S., & Watson, S. L. (2020). A Systematic Review of Asynchronous Online Discussions in Online Higher Education. American Journal of Distance Education, 1-14. https://doi.org/10.1080/08923647.2020.1858705

Gao, F., Zhang, T., & Franklin, T. (2013). Designing asynchronous online discussion environments: Recent progress and possible future directions. British Journal of Educational Technology, 44(3), 469-483. https://doi.org/10.1111/j.1467-8535.2012.01330.x

García-Álvarez, M. T., Novo-Corti, I., & Varela-Candamio, L. (2018). The effects of social networks on the assessment of virtual learning environments: A study for social sciences degrees. Telematics and Informatics, 35(4), 1005-1017. https://doi.org/10.1016/j.tele.2017.09.013

Garcia-Garcia, F. J., Moctezuma-Ramírez, E., Molla-Esparza, C., & López-Francés, I. (2021). Strategies based on social network analysis for enhancing the learning climate at universities. Research in Education and Learning Innovation Archives, 27, 33-46. https://doi.org/10.7203/realia.27.18960

Greco, P., & Piaget, J. (1959). Apprentissage et connaissance. P.U.F.

Habermass, J. (1984). Theory of Communicative Action. Volume One: Reason and the Rationalization of Society. Beacon Press.

Hammond, M. (2019). A Review of Recent Papers on Online Discussion in Teaching and Learning in Higher Education. Journal of Asynchronous Learning Networks, 9(3), 9-23. https://doi.org/10.24059/olj.v9i3.1782

He, X., & Meghanathan, N. (2016). Correlation of Eigenvector Centrality to Other Centrality Measures: Random, Small-World and Real-World Networks. Proceedings of the 8th International Conference on Networks and Communications (NeCoM), 9-18. https://doi.org/10.5121/csit.2016.61202

Hew, K. F., & Cheung, W. S. (2008). Attracting student participation in asynchronous online discussions: A case study of peer facilitation. Computers and Education, 51(3), 1111-1124. https://doi.org/10.1016/j.compedu.2007.11.002

Hülsmann, T., & Shabalala, L. (2016). Workload and interaction: Unisa’s signature courses – a design template for transitioning to online DE? Distance Education, 37(2), 224-236. https://doi.org/10.1080/01587919.2016.1191408

Jan, S. K., & Vlachopoulos, P. (2019). Social Network Analysis: A Framework for Identifying Communities in Higher Education Online Learning. Technology, Knowledge and Learning, 24(4), 621-639. https://doi.org/10.1007/s10758-018-9375-y

Jeong, A., & Chiu, M. M. (2020). Production blocking in brainstorming arguments in online group debates and asynchronous threaded discussions. Educational Technology Research and Development, 68, 3097-3114. https://doi.org/10.1007/s11423-020-09845-7

Jivet, I., Scheffel, M., Schmitz, M., Robbers, S., Specht, M., & Drachsler, H. (2020). From students with love: An empirical study on learner goals, self-regulated learning and sense-making of learning analytics in higher education. The Internet and Higher Education, 47. https://doi.org/10.1016/j.iheduc.2020.100758

Junus, K., Suhartanto, H., R-Suradujono, B. S. H., Santoso, H. B., & Sadita, L. (2019). The Community of Inquiry Model Training Using the Cognitive Apprenticeship Approach to Improve Students’ Learning Strategy in the Asynchronous Discussion Forum. The Journal of Educators Online, 16(1), 1-17. https://doi.org/10.9743/jeo.2019.16.1.7

Lahuerta-Otero, E., Cordero-Gutiérrez, R., & Izquierdo-Álvarez, V. (2019). Using Social Media to Enhance Learning and Motivate Students in the Higher Education Classroom. In L. Uden, D. Liberona, G. Sanchez, & S. Rodríguez-González (Eds.), Communications in Computer and Information Science (pp. 351-361). Springer. https://doi.org/10.1007/978-3-030-20798-4_30

Law, J., Barny, D., & Poulin, R. (2020). Patterns of peer interaction in multimodal L2 digital social reading. Language Learning and Technology, 24(2), 70-85. https://doi.org/10125/44726

Lee, D., Rothstein, R., Dunford, A., Berger, E., Rhoads, J. F., & DeBoer, J. (2021). “Connecting online”: The structure and content of students’ asynchronous online networks in a blended engineering class. Computers & Education, 163, 1-18. https://doi.org/10.1016/j.compedu.2020.104082

Lin, Y., Yu, R., & Dowell, N. (2020). LIWCs the Same, Not the Same: Gendered Linguistic Signals of Performance and Experience in Online STEM Courses. In I. I. Bittencourt, M. Cukurova, K. Muldner, R. Luckin, & E. Millán (Eds.), Artificial Intelligence in Education. 21st International Conference, AIED 2020, Ifrane, Morocco, July 6-10, 2020, Proceedings, Part I (pp. 333-345). Springer. https://doi.org/10.1007/978-3-030-52237-7_27

Maguire, R., Egan, A., Hyland, P., & Maguire, P. (2017). Engaging students emotionally: the role of emotional intelligence in predicting cognitive and affective engagement in higher education. Higher Education Research and Development, 36(2), 343-357. https://doi.org/10.1080/07294360.2016.1185396

Mazzolini, M., & Maddison, S. (2007). When to jump in: The role of the instructor in online discussion forums. Computers and Education, 49(2), 193-213. https://doi.org/10.1016/j.compedu.2005.06.011

Müller, F. A., & Wulf, T. (2020). Technology-supported management education: a systematic review of antecedents of learning effectiveness. International Journal of Educational Technology in Higher Education, 17, 1-33. https://doi.org/10.1186/s41239-020-00226-x

Negre, C. F. A., Morzan, U. N., Hendrickson, H. P., Pal, R., Lisi, G. P., Loria, J. P., Rivalta, I., Ho, J., & Batista, V. S. (2018). Eigenvector centrality for characterization of protein allosteric pathways. Proceedings of the National Academy of Sciences, 115(52), E12201-E12208. https://doi.org/10.1073/pnas.1810452115

Newman, M. E. J. (2010). Networks: An Introduction. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199206650.001.0001

Onyema, E. M., Deborah, E. C., Alsayed, A. O., Noorulhasan, Q., & Sanober, S. (2019). Online Discussion Forum as a Tool for Interactive Learning and Communication. International Journal of Recent Technology and Engineering, 8(4), 4852-4859. https://doi.org/10.35940/ijrte.D8062.118419

Pulford, B. D. (2011). The influence of advice in a virtual learning environment. British Journal of Educational Technology, 42(1), 31-39. https://doi.org/10.1111/j.1467-8535.2009.00995.x

Saadatdoost, R., Sim, A. T. H., Jafarkarimi, H., & Mei Hee, J. (2015). Exploring MOOC from education and Information Systems perspectives: a short literature review. Educational Review, 67(4), 505-518. https://doi.org/10.1080/00131911.2015.1058748

Sachdeva, C., & Gilbert, S. J. (2020). Excessive use of reminders: Metacognition and effort-minimisation in cognitive offloading. Consciousness and Cognition, 85, 1-14. https://doi.org/10.1016/j.concog.2020.103024

Sanganyado, E., & Nkomo, S. (2018). Incorporating Sustainability into Engineering and Chemical Education Using E-Learning. Education Sciences, 8(2), 1-11. https://doi.org/10.3390/educsci8020039

Saqr, M., Viberg, O., & Vartiainen, H. (2020). Capturing the participation and social dimensions of computer-supported collaborative learning through social network analysis: which method and measures matter? International Journal of Computer-Supported Collaborative Learning, 15(2), 227-248. https://doi.org/10.1007/s11412-020-09322-6

Schmitz, B., & Hanke, K. (2021). Engage me: Learners’ expectancies and teachers’ efforts in designing effective online classes. Journal of Computer Assisted Learning. https://doi.org/10.1111/jcal.12636

Sellke, T., Bayarri, M. J., & Berger, J. O. (2001). Calibration of ρ Values for Testing Precise Null Hypotheses. The American Statistician, 55(1), 62-71. https://doi.org/10.1198/000313001300339950

Silk, M. J., Croft, D. P., Delahay, R. J., Hodgson, D. J., Boots, M., Weber, N., & McDonald, R. A. (2017). Using Social Network Measures in Wildlife Disease Ecology, Epidemiology, and Management. BioScience, 67(3), 245-257. https://doi.org/10.1093/biosci/biw175

Smet, M. De, Keer, H. Van, Wever, B. De, & Valcke, M. (2010). Cross-age peer tutors in asynchronous discussion groups: Exploring the impact of three types of tutor training on patterns in tutor support and on tutor characteristics. Computers & Education, 54(4), 1167-1181. https://doi.org/10.1016/j.compedu.2009.11.002

Stephens, G. C., Rees, C. E., & Lazarus, M. D. (2019). How does Donor Dissection Influence Medical Students’ Perceptions of Ethics? A Cross-Sectional and Longitudinal Qualitative Study. Anatomical Sciences Education, 12(4), 332–348. https://doi.org/10.1002/ase.1877

Sun, J., & Tang, J. (2011). A Survey of Models and Algorithms for Social Influence Analysis. In C. C. Aggarwal (Ed.), Social Network Data Analytics (pp. 177-214). Springer. https://doi.org/10.1007/978-1-4419-8462-3_7

Thomas, J. (2013). Exploring the use of asynchronous online discussion in health care education: A literature review. Computers and Education, 69, 199-215. https://doi.org/10.1016/j.compedu.2013.07.005

Tirado Morueta, R., Maraver López, P., Hernando Gómez, Á., & Harris, V. W. (2016). Exploring social and cognitive presences in communities of inquiry to perform higher cognitive tasks. The Internet and Higher Education, 31, 122-131. https://doi.org/10.1016/j.iheduc.2016.07.004

Valente, T. W., Coronges, K., Lakon, C., & Costenbader, E. (2008). How Correlated Are Network Centrality Measures? Connections (Toronto, Ont.), 28(1), 16-26. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2875682/

Vázquez-Cano, E., Meneses, E. L., & Sánchez-Serrano, J. L. S. (2015). Analysis of social worker and educator’s areas of intervention through multimedia concept maps and online discussion forums in higher education. Electronic Journal of E-Learning, 13(5), 333-346. https://academic-publishing.org/index.php/ejel/article/view/1936

Vovk, V. G. (1993). A Logic of Probability, with Application to the Foundations of Statistics. Journal of the Royal Statistical Society: Series B (Methodological), 55(2), 317-341. https://doi.org/10.1111/j.2517-6161.1993.tb01904.x

Vygotsky, L. S. (1978). Mind in Society. The Development of Higher Psychological Processes. Harvard University Press. https://doi.org/10.2307/j.ctvjf9vz4

Wang, P. Y., & Yang, H. C. (2012). Using collaborative filtering to support college students’ use of online forum for English learning. Computers and Education, 59(2), 628-637. https://doi.org/10.1016/j.compedu.2012.02.007

Yapici, İ. Ü., & Akbayin, H. (2012). The Effect of Blended Learning Model on High School Students’ Biology Achievement and on their Attitudes towards the Internet. The Turkish Online Journal of Educational Technology, 11(2), 228-237. http://www.tojet.net/articles/v11i2/11224.pdf

Yoo, J., & Kim, J. (2012). Predicting Learner’s Project Performance with Dialogue Features in Online Q&A Discussions. In S. A. Cerri, W. J. Clancey, G. Papadourakis, & K. Panourgia (Eds.), Intelligence Tutoring Systems. 11th International Conference, ITS 2012 Chania, Crete, Greece, June 14-18, 2012 Proceedings. Springer. https://doi.org/10.1007/978-3-642-30950-2_74

Yoo, J., & Kim, J. (2014). Can Online Discussion Participation Predict Group Project Performance? Investigating the Roles of Linguistic Features and Participation Patterns. International Journal of Artificial Intelligence in Education, 24(1), 8-32. https://doi.org/10.1007/s40593-013-0010-8

Zou, W., Hu, X., Pan, Z., Li, C., Cai, Y., & Liu, M. (2021). Exploring the relationship between social presence and learners’ prestige in MOOC discussion forums using automated content analysis and social network analysis. Computers in Human Behavior, 115, 1-17. https://doi.org/10.1016/j.chb.2020.106582