Bodong Chen

Crisscross Landscapes

Notes: Lin. (2013). Online formative assessments with social network awareness



Citekey: @Lin2013

Lin, J.-W., & Lai, Y.-C. (2013). Online formative assessments with social network awareness. Computers & Education, 66, 40–53. doi:10.1016/j.compedu.2013.02.008


This paper reports an interesting SNA-based system that recommend peer helpers based on friendship statuses (‘social context’ as called by the authors) and results of ‘formative assessment’ (i.e., results of a formative test; ‘knowledge context’ as called by the authors). The tool, SNAFA, engages students in 1) self-report information including friendship statuses, 2) take a test, and 3) review recommended peer helpers to seek help. The design is novel in that it combines both social and knowledge contexts. (However, the results might be a bit thin because some causal claims made in this paper might fit better with correlation.)


his study focuses on being aware both peer’s social context and knowledge context for student to promote the opportunity of peer interaction and to select the appropriate helpers to ask for help when facing problems in online assessments. (p. 40)

A corresponding system, called Social Network Awareness for Formative Assessments (SNAFA), is further developed. The education experiments particularly focused on the effects of social-context awareness on learning activity and social activity. The results showed that the SNAFA not only increase the participant rate of students on formative assessment and opportunities of knowledge sharing, but also promote learning achievement, compared to the Traditional Formative Assessment (TFA). (p. 40)

  1. Introduction (p. 40)

In general, providing feedback on formative assessments is difficult for teachers because they face a large number of students, lengthy pieces of work, or practical constraints such as time and workload (Buchanan, 2000; Wang, 2007). (p. 40)

SNA is used to be aware of the knowledge context and social context of the others (Zheng & Yano, 2007). Knowledge context, which reveals the peer candidate’s knowledge expertise and experience, is used to match ‘who knows what’ in the knowledge dimension. Social context, which reveals the nimbus (i.e. willingness to help other) of the peer candidates, their social network tie, and social network position (i.e. central/peripheral positions in a network), is used to harmonize ‘who is willing to collaborate. Being aware of both context information and the interactions among peers are deemed as scaffolds at stimulating students to reflect on learning, to imitate peers’ actions, and to motivate coherent discussions. (p. 41)

However, most SNA studies either focus on being aware of peer’s knowledge context (El-Bishouty et al., 2007; Yang, Chen, Kinshuk, & Chen, 2007) or on social context (Cadima et al., 2010; Chen, Hong, & Chang, 2008). (p. 41)

This work proposes online formative assessments with social network awareness to raise the opportunities of collaborative learning and participant rate within a community (p. 41)

  1. Background (p. 41)

2.2. Social network awareness (SNA) (p. 41)

Social network awareness (SNA) has been extensively used as one of the strategies to promote the opportunities of peer interaction and collaboration, and to support the co-construction of knowledge and the sharing of information (Cadima et al., 2010; Chen et al., 2008; Chen, Chen & Kinshuk, 2009; Dawson, 2008; El-Bishouty et al., 2007; Hu et al., 2002; Yang, Chen, & KinshukChen, 2007). (p. 41)

However, most SNA related works focus on the awareness of either peer’s knowledge context (El-Bishouty et al., 2007) or peer’s social context (Cadima et al., 2010; Chen et al., 2008). The social-context studies mainly focus on being aware of peer’s social status, including social network tie, social network position, and social interaction. (p. 41)

Chen et al. (2008) presented a method of mining social interactive networks, which used a prediction model based on peer’s past social interaction (i.e. request and response messages), to recommend appropriate peer helpers for a requester. (p. 42)

On the other hand, the knowledge-context studies mainly focus on being aware of peer’s proficiency level of a specific domain, including knowledge level and experience. For example, El-Bishouty et al. (2007) proposed a ubiquitous learning system, which used a prediction model mainly based on candidates’ proficiency level to recommend the qualified candidate for a help seeker. (p. 42)

Yang et al. (2007) has proposed a prediction model to recommend candidates based on a peer’s trust association and knowledge association. The knowledge association is the level of proficiency of a candidate pertaining to a specific knowledge domain. The trust association is the trustworthy level of a seeker to a candidate, which is determined by past social interaction (i.e. request and response messages) experience. (p. 42)

2.3. Investigation in social networks (p. 42)

social network tie has a positive correlation with the intention of knowledge sharing (Chen et al. 2009) and has an impact on learning outcomes on student sense of belonging to a community (Haythornthwaite, 2002). Members connected via weak social ties are less likely to utilize and share resources and information in contrast to members connected via strong ties (Dawson, 2008). (p. 42)

In contrast, studies examining the impacts of social network position are relatively fewer and scant (Cho et al., 2007; Nurmela, Lehtinen, & Palonen, 1999). Cho et al. (2007) studies focused on the impact of social network position on learning performance in an online collaborative system. Nurmela et al. (1999) investigated only the relationship between social network position and message flows (received and sent messages) in an online collaborative system. (p. 42)

2.4. Contributions in this work (p. 42)

In summary, this work has three main contributions: (1) We propose the SNAFA, an online formative assessment system with social network awareness, which particular emphasizes on social-context awareness. (2) We compare the SNAFA and the Traditional Formative Assessment (TFA) and verify the superiority of SNAFA. (3) Many factors in social networks, such as message flow and system utilization, are further investigated via the SNAFA system to more perceive their effects on learning performance. (p. 42)

  1. The proposed SNAFA system (p. 43)

3.1. The overall process (p. 43)

The overall process consists of three phases: Initialization, Test, and Review phases. At the beginning, the Initialization phase is used to let each student preset his/her existing friendship network. Afterward, students enter the Test phase to take their online formative assessments. After the Test phase, students will know what questions they incorrectly answered. Then in the Review phase, students can know both social and knowledge contexts of peers and ask the desired candidates for help by referring the above information when encountering problems (p. 43)

In the Initialization phase, each student in a community registers his/her personal information (e.g. account, name, and photo) in the SNAFA and then sets his/her existing friendship by selecting his/her close friends. After this phase, the SNAFA will record the relationship between students into the Social Network Database for generating social network position in the Review phase. (p. 43)

The procedure of the Review phase consists of the following steps: Step 1: If all incorrectly answered questions (IAQs) are addressed, the Review process in this assessment terminates. Otherwise, the student enter step 2. Step 2: A student can ask for help by sending message to a helper. A student can select an appropriate helper though the provided information about social and knowledge contexts. (p. 43)

For each incorrect question, the student can click the corresponding button called “looking for peer assistance” to generates the candidate list. These candidates are all answering the question correctly. Most importantly, the information about knowledge context and social context of each candidate, which is elaborated in next subsection, are also displayed. Then, the student can select desired helpers from the list and send help messages. When a request has responded, the student (i.e. help seeker) receives a notification e-mail sent from the system (p. 43)

interesting and smart design :) (p. 43)

3.2. Constructing social network and generating social and knowledge contexts (p. 44)

Fig. 2. The process of constructing social network. (p. 44)

3.3. Graphic user interfaces (p. 45)

For any student, his/her corresponding ego-centered network is established while for a teacher, the whole social network is established (p. 45)

in which the blue ellipse indicates the student while other ellipses indicate peers who have close friendships with the student. (p. 45)

When a student selects an incorrectly answered question, the candidate list, which contains student ID, name, SD, DC, CC, WD, and KD, appears, as Fig. 4(b) shows. The student can sort the list according to these fields: SD, DC, CC, WD, and KD. (p. 46)

  1. Method (p. 46)

This research adopted a quasi-experimental design method. The study was administered to two 3rd-year classes of a university: the first class consisted of 53 students using SNAFA; the second class consisted of 52 students using TFA. (p. 47)

4.2. Measures (p. 47)

4.2.1. Learning performance (p. 47)

4.2.2. Participant social and learning activities (p. 48)

  1. Results (p. 48)

Appropriate peer helpers who have just learned something are often better able than teachers to explain it to their classmates in a way that is accessible (Nicol et al., 2006). Peer interaction also exposes students to alternative perspectives on problems and to alternative tactics and strategies. Alternative perspectives enable students to revise or reject their initial hypothesis, and construct new knowledge and meaning through negotiation (Nicol et al., 2006). (p. 48)

In the third row of Table 5, the number of asking messages for five formative assessments is 28, 30, 43, 60, and 56, respectively. Apparently, the stably-increasing number (p. 48)

5.2.2. Centrality versus posttest (p. 50)

Table 6 shows that the mean of the posttest of the Degree-HG (mean 14 76.27) is significantly higher than that of the Degree-LG (mean 14 63.23) (t 14 2.35, p < .05). It also shows that the mean of the posttest of the Closeness-HG (mean 14 73.96) is higher than that of the Closeness -LG (mean 14 65.54) (t 14 1.47, p > .05) although it does not reach significance. These results unveil that in the SNAFA, centrality indeed has certain influences on the posttest. (p. 50)

5.2.3. Centrality versus message flow Table 7 shows that the mean of the out-degree of the Degree-HG (mean 14 7.58) is significantly higher than that of the Degree-LG (mean 14 0.89) (t 14 3.24, p < .05). Also, the mean of the in-degree of the Degree-HG (mean 14 7.69) is significantly higher than that of the Degree-LG (mean 14 0.78) (t 14 2.77, p < .05). (p. 50)

These two results might be problematic, because two measures in each analysis are inter-dependent. For instance, centrality scores are linked to message flows (right?). (p. 50)

5.2.4. Centrality versus utilization (p. 51)

5.2.5. Correlation analysis (p. 51)

Table 11 shows the results and reveals that both centrality indices have a significant positive correlation with these variables: message flow (i.e. in-degree and out-degree) and utilization (i.e. retention time, and access time). Degree centrality has significant positive correlation with the posttest (the correlation 14 0.315, p < .01) but the correlation between closeness centrality and the posttest do not reach significance. (p. 51)

  1. Conclusion (p. 51)

In this work, social-context information particularly includes nimbus, centrality, and social distance. Knowledge and social-context awareness regards as a scaffold, trying to raise the opportunities of peer interaction and collaborative feedback within a community. (p. 52)

Sadler, D. R. (1998). Formative assessment: revisiting the territory. Assessment in Education, 5(1), 77–84. (p. 53)