Bodong Chen

Crisscross Landscapes

Notes: Sinha (2015) Collaborative group engagement in a computer-supported inquiry learning environment



Citekey: @Sinha2015

Sinha, S., Rogat, T. K., Adams-Wiggins, K. R., & Hmelo-Silver, C. E. (2015). Collaborative group engagement in a computer-supported inquiry learning environment. International Journal of Computer-Supported Collaborative Learning, 10(3), 273–307. doi:10.1007/s11412-015-9218-y



the quality of engagement (p. 273)

the quality of behavioral and social engagement differentiated groups demonstrating low quality engagement, but cognitive and conceptual-to-consequential forms are required for explaining high quality engagement. (p. 273)

In CSCL settings, the extent to which collaboration is productive in ways that lead to conceptual understanding depends on high quality engagement in shared activity. (p. 274)

Extant research and operationalization limit our understanding of deep-level engagement in CSCL contexts. Existing studies have operationalized engagement as a single facet, yielding a narrow view of engagement and the interaction among these facets (i.e., behavioral, emotional, cognitive) (Fredricks et al. 2004; Ryu and Lombardi 2015). (p. 274)

A second issue is that engagement has typically been evaluated at a single time point, with limited information provided about its evolving nature during task activity and over time. In addition, survey and observational measures of engagement evaluate individual learners, rather than shared engagement (Fredricks et al. 2004). (p. 274)

Finally, engagement has primarily been operationalized as general in regards to the task and classroom context, providing a decontextualized understanding of engagement. (p. 274)

This research is conducted in the context of middle school students learning about ecosystems (Eberbach et al. 2012; Jordan et al. 2013, 2014). This research is particularly timely as systems are key crosscutting concepts in current science standards (NGSS 2013), but remain challenging for learners because of the dynamic multi-level nature of systems (HmeloSilver and Azevedo 2006; Hmelo-Silver et al. 2007). (p. 275)

Building from Gresalfi’s notion of consequential engagement (Gresalfi et al. 2009), we use conceptual-to-consequential engagement to reflect how groups engage with ideas such that their application has consequences for solving a contextualized problem in a CSCL environment. (p. 275)

Engagement in CSCL environments (p. 275)

Engagement is central to understanding how to foster conceptual understanding because engagement mediates the relationship between motivation and learning (Blumenfeld et al. 2006). (p. 275)

Our guiding theoretical framework is consistent with Fredricks et al. (2004) in three primary ways. First, we consider engagement to be a multi-faceted construct that unites varying forms of engagement in meaningful ways as a Bmeta-construct^ (Fredricks et al. 2004, p. 60). (p. 275)

Second, we assume that there are qualitative differences in the degree of engagement for each component. (p. 276)

Finally, engagement is responsive to context, with the specific context encompassing the CSCL environment, acknowledging that this immediate learning context is necessarily nested within a larger classroom, instructional, and curricular context. (p. 276)

Accordingly, we need to build from this conceptualization to account for the group as the unit of analysis. A consideration of collective engagement necessitates an examination of social interactions among students and the shared nature of their engagement. (p. 276)

In what follows, we define each dimension of engagement, and review research related to engagement in collaborative groups and/or conducted within CSCL environments. (p. 276)

Behavioral engagement Behavioral engagement involves sustained on-task behavior during academic activity, including indicators such as persistence, effort, and contributing to the task (Fredricks et al. 2004). (p. 276)

Individual learners who withdraw their participation from group discussion can undermine learning, due to lost opportunities for collaboration or by provoking whole group disengagement (Van den Bossche et al. 2006). (p. 276)

Consistent with this definition, studies of student engagement within CSCL have primarily employed measures consistent with this engagement facet, measuring students’ participation given number of contributions (Lipponen et al. 2003), length of posts in online environments (Guzdial and Turns 2000), or whether contributions are more social (i.e., off-task) rather than around content ideas (Stahl 2000). (p. 276)

Social engagement Our inclusion of social engagement extends beyond the behavioral, emotional, and cognitive engagement distinctions made by Fredricks and her colleagues (Fredricks et al. 2004) to account for social interactions within small groups. Drawing from Linnenbrink-Garcia et al. (2011), we define social engagement as referring to quality of group socio-emotional interaction.1 Quality social engagement involves respectful and responsive interactions among members of the group. Social engagement also reflects group cohesion, or evidence that the task is conceptualized as a team effort, rather than an as an individual activity. (p. 276)

Finally, quality social interactions reflect equitable participation in which all teammates contributions are taken up (Barron 2003; Rogat and Adams-Wiggins 2015). (p. 277)

This research highlights that groups often face difficulty finding common ground and may lack shared understanding (Dillenbourg et al. 2009). Negative social interactions can come to predominate group activity, and compete for limited attentional resources (Barron 2003). In worst cases, low quality social interactions can devolve into battles related to status differences and can promote inequity (Salomon and Globerson 1989). (p. 277)

respectful, responsive, and cohesive interactions elevate the quality of joint task work (Engle and Conant 2002; Webb et al. 2006). Further, positive social interactions can facilitate higher quality cognitive engagement by helping ensure that feedback from monitoring was communicated well, supported joint and inclusive planning (Rogat and Linnenbrink-Garcia 2011). (p. 277)

Cognitive engagement Fredricks and her colleagues (2004) indicate that there are two primary conceptualizations of cognitive engagement, both in terms of investment in schooling (e.g., Connell and Wellborn 1990) as well as being a strategic and self-regulated learner (e.g., Pintrich and De Groot 1990). (p. 277)

Socially shared regulation refers to multiple group members regulating and coordinating their joint activity (Vauras et al. 2003). Rogat and Linnenbrink-Garcia (2011) used the cognitive (p. 277)

sub-processes from a self-regulated learning perspective to understand and elaborate the quality variation of collaborative groups engaging in shared planning and monitoring. (p. 278)

Limited research has investigated how groups effectively regulate within CSCL environments (Järvelä and Hadwin 2013). We know that computersupported learning can support and enhance students’ use of regulatory processes (Azevedo 2005). Socially shared regulation research has demonstrated the presence of frequent and at times extended use of regulatory processes within synchronous and asynchronous CSCL environments (Iiskala et al. 2011; Lee et al. 2015). (p. 278)

groups exhibiting moderate to low quality cognitive engagement during planning or monitoring may demonstrate a focus on superficial features, such as brief planning discussions or a focus on color or neatness, with implications for challenges reaching consequential engagement via the technology tools. (p. 278)

Conceptual-to-consequential engagement Our introduction of conceptual-to-consequential (CC) engagement provides an important extension to the forms synthesized in Fredricks et al. review (2004). CC engagement refers to making progress in solving meaningful problems through the use of domain-specific content and disciplinary practices as conceptual tools (Gresalfi and Barab 2010; Gresalfi et al. 2009). (p. 278)

Consequential engagement also specifies an active and agentic role for learners to justify identified solutions, particularly after having weighed and critiqued alternative solutions to the problem. In this way, consequential engagement builds from the connections and synthesis, as well as regulation, from cognitive engagement, toward a reflection of connecting to something larger. (p. 278)

Extant research has suggested that students’ connections between conceptual ideas and a broader context can be lower quality, shown by simple knowledge telling with limited connections (Scardamalia and Bereiter 1996; Chernobilsky et al. 2004), to moderate quality showcasing connections among content ideas (conceptual engagement or sense making), to higher quality linkages among content ideas with prior knowledge, everyday experiences, and/or the context of the larger problem (i.e., consequential engagement). Thus, we extend this construct by including conceptual engagement as part of a continuum that should culminate in consequential engagement. (p. 278)

They note that contexts and practices that Bemphasize making connections can only lead to robust learning when they are supported by tasks that create opportunities for students to grapple with the meaning and utility of content^ (Gresalfi and Barab 2010, p. 301). (p. 278)

Current study (p. 279)

This paper examines a multi-faceted, dynamic, shared, and contextualized conceptualization of engagement within a CSCL environment using our newly developed observation protocol. Toward this end, we initially explored quality variation in ten collaborative groups’ behavioral, social, cognitive and conceptual-to-consequential engagement using quality ratings. This was followed by coupling in-depth qualitative analysis and contrasting cases of two groups characterized by high or low quality engagement relative to the sample with the intent of describing engagement quality, the fluctuation in engagement quality during the lesson, and the interrelationships among engagement facets. In developing these cases, we prepared narratives that thickly described each engagement dimension in 5-minute intervals and visual representations of each group’s engagement ratings across a lesson. We also examined each group’s final explanatory model that was the subject of the observed lesson. A final analytic focused on case group comparison. (p. 279)

Research Question: How does a multifaceted, shared, dynamic and situated conceptualization of engagement serve as an observational tool for studying CSCL? (p. 279)

Method (p. 279)

Students participated as part of a technology-intensive curriculum designed to support 7thgraders’ learning about aquatic ecosystems (Hmelo-Silver et al. 2011). (p. 279)

For this study we focused on the pond unit, with the driving question on investigating causes for fish death in a local pond. (p. 279)

Classroom instruction was a mix of whole class and small group activities organized around components-mechanisms-phenomena (CMP). CMP is a conceptual representation adapted from Structure-Behavior-Function theory (Vattam et al. 2011; Hmelo-Silver et al. 2007; see also Quellmalz et al. 2009). In brief, phenomena are the problems or patterns to be explained (here, the sudden fish death in the pond). Components are the individual entities in the system (e.g., fish), and mechanisms are characterized as causal explanations of how phenomena occur or how significant processes work (e.g., cellular respiration). (p. 280)

Technologies (p. 280)

Simulations, modeling tools and hypermedia were an integral part of the curriculum that promoted the usage of CMP as a conceptual tool to make sense of problems in the aquatic ecosystem. (p. 280)

Ten videotaped collaborative groups are the focus of this research. (p. 280)

Each group consisted of three to four students. (p. 280)

Measures (p. 281)

we developed an observation protocol designed to evaluate collaborative group engagement using four dimensions (p. 281)

For each engagement dimension, a rating of low, moderate, or high (range 1–3) was assigned to reflect quality of group engagement (see Table 1). (p. 283)

Observations were segmented into 5-minute intervals, beginning when collaborative group activity was initiated (i.e., excluding teacher directions; whole class discussion). (p. 283)

Data analysis (p. 284)

In a second set of qualitative analyses, we sought to construct a rich description of collaborative groups’ engagement quality when working with the technology tools, with three primary emphases. First, we aimed to differentiate low and high quality collaborative engagement using thick descriptions through the analysis of two groups. Second, we aimed to explore the interrelationships among engagement dimensions in regards to their reciprocal influence during group interactions. Finally, we sought to analyze how engagement quality and the interrelations among dimensions unfolded over the course of group activity during a lesson. (p. 284)

Results (p. 286)

Engagement quality across groups (p. 286)

Correlations (p. 286)

First, although behavioral engagement was correlated with social and cognitive engagement, it had only a moderate correlation with conceptual-to-consequential engagement. (p. 286)

CC engagement was significantly related to cognitive engagement and moderately related to social engagement. (p. 286)

Mean engagement scores (p. 286)

these descriptive statistics illuminated between-group differences, suggesting substantial quality variation in engagement across the ten groups. (p. 286)

We drew on the means to identify groups 6 and 10 for follow-up case analyses (p. 286)

Quality of collaborative group engagement: contrasting cases (p. 287)

Behavioral engagement Group 6 frequently engaged in off-task conversations, reflecting a decision for only one group member to use the EMT software and add to the model at a time. (p. 288)

Social engagement Social engagement during model creation and revision was primarily moderate. (p. 289)

This use of BI^ shows a focus on individual thinking, and a conceptualization of the work as individual activity, rather than collaborative. (p. 289)

Another indicator of low quality social engagement was that even when group members’ contributions were solicited and/or shared, these ideas were not consistently taken up for discussion or further incorporated into the group model. (p. 290)

Taken together, low quality planning yielded activity that focused on listing factors responsible for low oxygen levels in the water, rather than developing an explanatory model for causes of sudden fish death. (p. 291)

Conceptual-to-consequential engagement (p. 292)

A fundamental challenge relevant to CC was that the group’s work on the modeling activity did not seem to connect to the larger unit problem explaining fish death in a pond, but remained narrowly focused on a single factor. (p. 292)

Taken together, Group 6 did not seem to use the available conceptual, scientific, and instructional resources in consequential ways to solve the larger problem of fish death in the local pond. (p. 292)

The separate examination of each engagement dimension was an important step in understanding the challenges faced by Group 6. In this final section, we consider how the mutual influence among dimensions explains Group 6’s lower quality engagement. (p. 292)

High quality engagement case (p. 293)

Group 10 displayed moderate-high level social engagement over the course of the lesson. The group’s social interaction can be characterized by Matt taking on a role in facilitating the group’s responses on their shared model. (p. 294)

In some instances, Matt employed the use of I when introducing his idea, but then returned to using Bwe^, suggesting some sensitivity to acknowledging the import of the collective or group. (p. 294)

This tension and difficulty with Matt’s perceived direction during group work resulted in some disrespectful exchanges marked by mimicking and ignoring. (p. 294)

Their planning discussion (p. 294)

This planning was in response to monitoring when the group was discussing more general causes of fish death based on their outside knowledge: (p. 295)

Conceptual-to-consequential engagement Group 10 can be differentiated by their maintained high quality CC engagement. (p. 295)

Beyond Group 10’s maintained focus on explaining fish death, this excerpt highlights that they consistently worked to ensure their model could be justified using the evidence drawn from the resources. (p. 297)

discussing mechanistic behaviors of those components in the context of the given problem. (p. 298)

First, this group showed consistent high-level behavioral and social engagement. It seems likely that because the group maintained on-task participation, they were able to maintain a shared focus on improving their conceptual artifact (the EMT model). Further, under Matt’s facilitating role within the group, group member’s contributions and perspectives were respected and considered for inclusion in the explanatory model. Here, both BE and SE seemed to be a critical undercurrent for reaching high levels of CE and CC. (p. 298)

The second pattern concerned the interrelated nature of CE and CC, with high quality cognitive engagement proving central to promoting and sustaining the group’s consequential engagement. (p. 298)

Discussion (p. 298)

Unpacking group engagement in CSCL contexts (p. 300)

We developed an observational protocol that operationalized engagement using four dimensions. (p. 300)

Our findings have implications for how we conceptualize the relations among forms of engagement. In particular, our results suggest interrelations among behavioral, social, and cognitive forms of engagement, with subsequent influence for groups’ CC engagement. (p. 301)

Here, we provide a review of Group 10’s case that builds toward these points. First, for Group 10, we see on-task participation and a positive climate as setting the stage for higher quality CE and CC engagement. Here, broad participation and sustained on-task engagement ensured mutual attention over the course of activity. Further, positive socio-emotional interactions, reflective of responsive interactions and the equitable solicitation of ideas, ensured that group member’s ideas were taken up and integrated within the group response. It is notable that the resulting positive interactions and inclusiveness required continued effort by Matt to ensure that there was agreement among group members related to the components and relationships integrated into their shared explanatory model. Future research should continue to examine the role of social engagement for engagement quality and more generally for group activity. (p. 301)

However, Group 10’s deep-level engagement was more than everyone’s participation and responsive interactions. It was our addition of CE and CC that enriched and elaborated our description of the deep-level engagement showcased by Group 10. Future research should explore the threshold at which behavioral and social engagement must be attained in order to sustain high quality CE and CC engagement. (p. 301)

However, we can also see consequential engagement as related to the synergistic influence of these engagement dimensions, (p. 301)

Taken together, our findings culminate in a synergistic view of engagement (also see Rogat and Linnenbrink-Garcia 2013). Our results give primacy to the highly interrelated and mutually influencing nature of these four dimensions of engagement. (p. 302)

Implications for design and instruction (p. 302)

Based on our findings, we present suggestions for refining the design of these technologies to promote groups’ cognitive and conceptual-to-consequential engagement. (p. 302)

Built-in prompts can appear on the screen when groups add new components or write explanations connecting two components. (p. 302)

However, designers need to carefully consider the conditions under which those prompts might appear to create a balance between encouraging thoughtfulness and interfering in the flow of the collaborative work. (p. 302)

Given that groups worked on model creation synchronously, it may have been a challenge for the teachers to monitor the conceptual and scientific practice understanding, as well as progress made by each group. (p. 302)

There is a general concern that schools do not give students opportunities to engage with curricular content in conceptually and consequentially meaningful ways (Gresalfi et al. 2009). (p. 302)

This study is important to the field of CSCL as it adds to the literature on inter-subjective meaning making (Koschmann et al. 2003, Koshmann et al. 2005, Rochelle 1996; Stahl 2004; Suthers 2006). (p. 303)

Conclusions CSCL environments are complex and attempts at understanding them need complex conceptualizations of how and whether groups take up the technological affordances in productive ways (e.g., Kapur et al. 2011; Teasley 2011). We argue that to richly conceptualize collaborative engagement in computer-supported contexts we need to draw on a multi-faceted, shared, and contextualized operationalization that extends beyond participation and group socio-emotional interactions. Our results show that these forms of engagement are interrelated and that the quality is mutually influential. Moreover, high conceptual-to-consequential (CC) engagement is facilitated by the synergistic influence of behavioral, social, and cognitive engagement dimensions. The CC dimension is especially important in computer-supported inquiry learning because we want learners to use the technology to go beyond building knowledge for its own sake (Chan 2013). Rather, the goal is knowledge building for action in which learners use knowledge as a tool for thinking (Hmelo et al. 1998). (p. 303)