Bodong Chen

Crisscross Landscapes

Notes: Barab. (2001). Constructing Networks of Action-Relevant Episodes: An In Situ Research Methodology

2017-06-16


References

Citekey: @Barab2001-jt

Barab, S. A., Hay, K. E., & Yamagata-Lynch, L. C. (2001). Constructing Networks of Action-Relevant Episodes: An In Situ Research Methodology. Journal of the Learning Sciences, 10(1-2), 63–112. https://doi.org/10.1207/S15327809JLS10-1-2_5

Notes

Summarize: This paper introduces a methodology that’s grounded in Activity Theory and situated cognition, and based on ethnographic approaches and network analysis. It offers a very thorough and useful exploration of the need for analysis of knowing ‘in situ’, compared with traditional analysis of learning results / knowledge products. The argument is sound.

The methodology itself is quite complicated and labor intensive. I found myself a bit lost when diving into detailed examples & trying to connect between them and earlier conceptual promises. This is due to the complexity of this methodology.

Reflect: This paper offers great insights in 1) analyzing learning in situ; 2) writing about a proposed methodology. It also share a common interest with my current work on DNA.

Highlights

In this article, we advance a methodology for capturing and tracing the emergence, evolution, and diffusion of a practice, conceptual understanding, resource, or stu- dent-constructed artifact. The Constructing Networks of Action-Relevant Episodes (CN–ARE) methodology allows researchers to identify relevant data from a complex, evolving environment, and then to organize it into a web of action that can illuminate the historical development (evolving trajectory) of the phenomenon of interest (e.g., conception of an eclipse, applications of a mathematical formula, an evolving stu- dent-constructed Website). (p. 2)

To accomplish this end, experiences are (a) sectioned into action-relevant episodes (AREs), (b) parsed down to codes in a database, and © then represented as nodes in a network so that the historical development of the particular phenomenon of interest can be traced. (p. 2)

this discussion includes an in-depth descrip - tion of the methodology along with its application to data sets. (p. 2)

Drawing primarily on the learner-as-processor metaphor, there is a long tradition of methodological practices for assessing the learning process (e.g., Sax, 1989; Wittrock & Baker, 1991). (p. 3)

Given the individual or, more specifically, the mind of the individual as unit of analysis these traditional methods appear to deal more or less adequately with capturing the phenomena of paradigmatic interest (Brown, 1992; Schoenfeld, 1992). However, as one moves to a distributed perspective it is the complex and dynamic intersection of individual, environment, and activity over time that constitutes the unit of analysis. The focus is on learning and knowing in practice, not on some hypothesized reification of practice. (p. 3)

However, a central methodological concern in our research has been how to capture the trajectory of learning as it unfolds over the se- mester-long courses. In other words, rather than describing students’ ready-made knowledge at the end of the course we have been interested in tracking knowing in the making as the course unfolds. (p. 3)

there have been few attempts to develop methodologies for mak - ing sense of how learner understandings are constructed and are grounded across contextual particulars that occur over extended time frames (see Roth, 1998, for an exception). In fact, research in general tends to look at the products, not the pro- cesses of learning (Wittrock & Baker, 1991; Young et al., 1997). (p. 3)

knowing about refers to a dynamic activity (trajectory of participation) that is dis- tributed across knower and that which is known and is spread out across extended time frames and multiple resources (Barab et al., 1999). (p. 4)

It is the participation trajectory over time that constitutes the unit of analysis when one adopts a situativity perspective of knowing and learning (Greeno, 1998). (p. 4)

it is distributed spatially and temporally across multiple components (p. 4)

It is in introducing a methodology for capturing and representing this “distributedness,” what we view as cognition, that this article is targeted. More specifically, in this ar- ticle we represent this trajectory as a network of activity—a network that allows for the inclusion (capturing) of material, individual, and social components over time. (p. 4)

Our methodology allows us to capture occurrences distributed across time and space that influence and constitute a learner’s understanding, providing information on how environ- mental particulars contribute to evolving understandings. (p. 4)

It is in this sense that we view cognition as distributed, embodied, and situated, and it is with the goal of capturing knowing in the making that we advance our Constructing Net- works of Action-Relevant Episodes (CN–ARE) methodology. (p. 4)

our first goal is to provide a description of the commitments and assumptions that underlie the design of our methodological approach. Our second goal is to describe this methodological approach. (p. 4)

Following a description of our methodologi - cal approach, the third goal of this manuscript is to show the application of the ap- proach to a particular data set, including touching on issues of trustworthiness, credibility, and usefulness. (p. 5)

THEORETICAL BACKGROUND (p. 5)

Cognition as Situated (p. 5)

Knowledge is not some ontological substance that lies in peoples’ heads (or in the pages of textbooks) waiting to be actualized through cog- nitive processes. Instead, and consistent with our relational or situated perspective, knowing is an action-relevant term that delineates a person’s potential to act in a certain fashion. (p. 5)

Briefly, knowing about 1. refers to an activity—not a thing. 2. is always contextualized—not abstract. 3. is reciprocally constructed as part of the individual–environment interac- tion—not objectively defined or subjectively created. 4. refers to functional relations—not objective “truths.” (p. 5)

in our conception, knowing about and learning are simply differ - ent ways of describing the dynamics of evolving participation. Becoming knowledgeably skillful, from this perspective, is characterized by an individual’s in- creasing potential to build and transform relations with the (material, psychological, and social) world. (p. 5)

nodding (p. 5)

Learning is thus fundamentally connected with and constitutive of the environ- mental particulars (including other people) through which it is actualized (Cobb & Yackel, 1996; Lave, 1997). Conceived in this fashion, the boundaries among indi- vidual cognition and the material and social world become difficult to identify. (p. 6)

In addition to stretching cognition across multiple individuals, context, activity, and the cultural and material artifacts of the world, cognition also is spread across multiple time frames (see Kulikowich & Young, 2001, this issue; Roth, 2001, this issue). (p. 6)

distributed across, the physical, temporal, and spatial occurrences through which her competencies have emerged. (p. 6)

To clarify the argument, if the goal is to account for the historical development of knowing, then the methodology must capture and coordinate the multiple inter- actions, distributed across time and space, in a fashion that constitutes the trajec- tory of knowing in the making. It is the complex and dynamic intersection of individual, context, and activity over time that constitutes the unit of analysis (Engeström, 1993; Greeno, 1998; Lave, 1988). (p. 6)

nodding (p. 6)

from a situative perspective, as - sessments and methodological approaches that focus solely on the individual learner are necessarily limited, and will fail to provide the rich contextual descrip- tions of knowing about that are so fundamental to situative or distributed concep- tions of cognition. (p. 6)

A Methodological Grounding (p. 7)

In advancing a new methodological approach, it is important to acknowledge intel- lectual history while at the same time illustrating where and why the approach be- ing advanced departs. (p. 7)

has close ties with and must acknowledge the intellectual contribution of (a) inter- action analysis, (b) network approaches, © Activity Theory, and (d) the notion of tracers. (p. 7)

Beginning with interaction analysis, our approach has much overlap with the work of Jordan and Henderson (1995). In their work, they bring together groups of researchers to analyze videos. The goal in their work is to develop coding schemes, whatever they may be, for segmenting and building interpretations of the episodes depicted in the video. (p. 7)

However, in our approach we have committed to a particular coding scheme and our focus is on the relations among episodes, not simply the episodes themselves. In this sense, our methodology can be considered one particular instance of interaction analysis in which we have committed to a particular set of assumptions that inform our de- scription of what constitutes an episode and how we make sense of the network of interactions that these episodes form. (p. 7)

network analysis was applied not just as a sociological exercise but as an analytical tool. (p. 7)

The methodology discussed in this article integrates a network approach for cap- turing the distributed and situated nature of knowing in the making. (p. 8)

In our methodology, the determination of what constitutes a node is informed by Activity Theory (Barab, Barnett, Squire, Yamagata-Lynch, & Keating, in press; Engeström, 1987, 1993, 1999; Leont’ev, 1974, 1989), (p. 8)

When referring to “activity,” activity theorists are not simply concerned with doing as disembodied action but are referring to doing to transform some object, with a focus on the contextualized activity of the system (Barab, in press; Engeström, 1987, 1993; Kuutti, 1996; Nardi, 1996). (p. 8)

The meditating components are tools (conceptual and physical), community, rules, and divisions of labor (Engeström, 1987, 1988, 1993; Kuttii, 1996). (p. 8)

For Activity Theory, the context is not simply a container nor a situationally created experiential space, but is an entire activity system, integrating the partici- pant, the object, the tools (and even communities and their rules and divisions of labor) into a unified whole (Barab, Barnett, et al., in press; Engeström, 1993). Sim- ilarly, activity is not one aspect of learning and learning is not one type of activity; activity is learning and learning is activity. (p. 8)

Activity Theory has much to offer in tackling the theoretical and methodological questions that are cen- tral to theories that suggest cognition as practice-bound or -situated; for example, situated cognition. We have found Engeström’s characterization of the elements that constitute an activity system to be informative in terms of conceptualizing the constituent features and the boundaries of a node in our network. (p. 9)

Consistent with Activity Theory (Engeström, 1987, 1993; Leont’ev, 1974; Vygotsky, 1978), each one of these AREs include participants who act on particu- lar objects (individual and groups), as well as other mediating components such as tools, resources, and other participants—all in relation to the participant’s goals and intentions. (p. 9)

Although nodes provide basic building blocks of our methodology, it is impor- tant to note that each component of a node (a participant’s understanding, a tool, an object being acted on), in addition to being a part of the current activity being ex- amined, is also constituted by previous instances of activity through which it was developed. For example, although a computer or a student-created inscription may be a tool in one network, on a previous occasion it may have been the object of ac- tivity (Barab, in press; Barab, Barnett, et al., in press; Latour, 1987). In other words, although nodes and their components exist in one network, nodes and their components are also constituted by networks; that is, nodes are both constitutive of and constituted by networks, reciprocally determining and being determined by the episodes in which they are a part. (p. 9)

Our desire to capture the historical development and diffusion of practices, con- ceptual understanding, resources, and artifacts has much overlap with how others have applied Actor–Network Theory. Actor–Network Theory is a sociological ap- proach developed by Callon and Latour (Callon, 1987; Callon & Latour, 1981; Latour, 1987) to trace the emergence, evolution, and diffusion of scientific knowl- edge and artifacts across a society. The network approach “allows researchers to position actors within a larger context and reflect on their specific ‘mediating’ roles and to formulate appropriate practices of intermediation” (Gartner & Wag- ner, 1996, p. 187). (p. 10)

Although we may follow the trajectory of a nonhuman artifact or even examine the relations of a nonhuman artifact and a particular understanding, in our conception only humans are coded as actors. Our focus is primarily on tracing the events through which an individual or a number of individuals come to engage in a specific practice, under- stand a particular concept, evolve their use of a resource, or construct a particular artifact. (p. 10)

Roth (1996) used network theory to examine how learning unfolds within stu- dent-centered classrooms. (p. 10)

Central to his research was the notion of tracers. Newman, Griffin, and Cole (1989) used the term tracer to denote a preexisting methodological strategy to find the “same activity” across different contexts. In our use, and consistent with the work of Roth (1996; Roth & Roychoudhury, 1993), tracers can refer to practices, conceptual understanding, and student productions (e.g., projects developed) that can be observed and fol- lowed over time. In Roth’s (1996) research, tracers were selected and then their history was followed through the network. (p. 10)

THE CONSTRUCTING NETWORKS OF ACTION-RELEVANT EPIDODES METHODOLOGY (p. 11)

Developing A Methodology (p. 11)

we situate our method- ological discussions using a simplistic hypothetical example and then describe ac- tual data collected as part of our previous research to further illuminate the process and usefulness of our methodology (p. 11)

Data Collection (p. 11)

Instead, data collection must also be situated in social interactions that are distributed (p. 11)

videotaping events provides a necessary historic re - cord of the event that (p. 12)

Eventually, these observations are chunked into episodes, assigned labels, and organized as part of a relational database (described more fully later). (p. 12)

In addition to direct classroom observation, we also use multiple video cameras that are directed at individual learning groups in a particular class- room so that the researcher can code and reanalyze “fast-flying” interactions (see Jordan & Henderson, 1995, for an in-depth description of shooting videotape for analysis). (p. 12)

To supplement and triangulate interpretations, our team also collects field notes, student-constructed artifacts, and carries out interviews with students and teachers. In particular, the data-collection efforts have been informed by other nat- uralistic accounts of classroom data-collection practices (Roth, 1996) (p. 12)

In constructing and triangulating interpretations, we use the multiple data sources; however, the primary data col- lection procedures are direct observation and video recordings. (p. 12)

Defining Ethnographic Chunks: The AREs (p. 12)

Operationally, the identification or “chunking” the raw data into meaningful units or nodes is the first step in the creation of the network. (p. 12)

In the ana - lytical tools of Event-State and Causal Networks (Miles & Huberman, 1984), the nodes are defined as time-dependent events that “happen” (a meeting, a conversa- tion, or a mouse click) or they are defined as a state of mind (student frustration or pressure by parents). (p. 13)

In our approach, nodes are equivalent to what qualitative re - searchers have described as “units,” “chunks of meaning,” or “ethnographic chunks” (Jordan & Henderson, 1995; Lincoln & Guba, 1985). (p. 13)

AREs are identified as activity occurrences that are judged to be a significant happening in the learning context, and are delimited by a change in theme, activity, subject, or resources. (p. 13)

What qualifies as a significant happening or a segment is somewhat subjective and spe- cific to the needs and interests of each particular research context. (p. 13)

Lincoln and Guba (1985) described two criteria for selecting units of analysis: They must be heuristic and they must be the smallest piece of information about something that can stand by itself. (p. 13)

TABLE 1 Summary Labels for the Various Features That Constitute a Node (p. 14)

Issue at hand A summary label that is chosen to identify the content of the node. It is the “direct” object of discussion or manipulation (the only way a practice can be considered an issue at hand is if it becomes the explicit object of discussion or manipulation). It can refer to an artifact, tool-related practice, or a conceptual tool or process. (p. 14)

Initiator An individual or group (when engaged in a practice as a single unit) that is producing an action. (p. 14)

Participant An individual who is involved in a node but not initiating the action. (p. 14)

Resource “Any piece of information, object, tool, or machine” that an initiator uses to carry out a practice (Roth, 1996, p. 191). In addition to technological tools, our definition of tool includes those of a conceptual nature (i.e., heat–color relations) and those of a social nature (e.g., community norms). An artifact is transformed to a resource when it is used by an actor as part of a practice. (p. 14)

Practice An activity that is carried out by an initiator who is using a resource. Practices can be tool related (i.e., embodied tool-related laboratory skills), scientific (i.e., calculating), instructional related (i.e., coaching), learning related (i.e., using an inquiry strategy), or conceptual (theorizing about quantum mechanics) and always involve the use of a resource. (p. 14)

In our research, whenever we observed a change in the focus of an epi - sode (e.g., from eclipses to animation), the practice (from modeling to Socratic questioning), or the participants (from one student to another) we coded a new epi- sode. (p. 15)

Defining the Core Categories (p. 15)

Consistent with Activity Theory, the issue at hand is the “object” of discussion or manipulation. (p. 15)

An initiator is an individual (the participant) or group (when engaged in a prac- tice as a single unit) that is producing an action. (p. 15)

Additionally, we have not included nonhuman initiators; however, the contribution of nonhu- man objects (tools and resources) becomes fused and is a part of those networks in which they played an important part. The coding of these nonhuman objects is not as initiator but as issue-at-hand or as resource. (p. 16)

the fourth category we identified as a critical element is the tool or re- source. A resource is “any piece of information, object, tool, or machine” that an initiator uses to carry out a practice (Roth, 1996, p. 191). (p. 16)

The final critical element of any ARE is the practice. The practice is an action carried out by an initiator or participant. There can be many different categories of practice within a network of AREs (N–AREs). Some of these categories of prac- tice are associated with specific types of initiators (p. 16)

However, each practice involves a specific resource (p. 16)

Ethnographic Descriptions (p. 17)

it has also proven necessary to add a field to our database for ethnographic descriptions that allow researchers to gain a rich contextual picture when later examining the nodes. (p. 17)

Building Tracer Networks of Activity (p. 17)

The second main feature of a network is the links that connect the nodes. We con- ceptualize the links as anything that ties one node (an ARE) to any other node. (p. 17)

Time links nodes his - torically, practices link nodes of similar practices together, resources link nodes of specific resources used together, and initiator and participant codes link people. These linkages through all the nodes of a given database can be envisioned as akin to a densely woven, highly complex “knot” of nodes and links. (p. 17)

The question becomes, What links are productive to graphically represent within a network to address the underlying questions? (p. 18)

In our work, to bring order to this knot, we have developed a method to visualize this database in a fashion that enables us to explore the issues around the emergence, evolution, and diffusion of practices, concepts, resources, and artifacts occurring over extended time frames. (p. 18)

Central to this research is the notion of tracers, which we use to denote those facts, practices, student productions, or understandings that can be observed and followed over time (Newman et al., 1989; Roth & Roychoudhury, 1993). In our approach, tracers are identified through grounded theory development (p. 18)

Visualizing Nodes and Links (p. 19)

Operationally, the visualization and representation of links starts with the visualization of the nodes on a time line. This process begins by developing graphs in which the y-axis represents ordinal, not necessarily ratio, time and the x-axis represents the node initiator. (p. 19)

These node bars are then abstracted into numbered circles, and positioned in the appropriate initiator column (see Figure 3). (p. 19)

Participants and links are then added to the network. Beginning with partici- pants, in Figure 4 noncircled numbers have been added to the network to indi- cate additional individuals who were not initiators to a particular node. (p. 21)

Continuing with an explanation of Figure 4, this visualization illustrates a network of a tracer with links representing the connection of two nodes that share common facts, practices, student productions, or understandings, and are related historically through a common actor. (p. 21)

In the aforementioned discussion, we described the process of constructing N–AREs. The process from observation to analysis involves the following steps: 1. Collection of the data through direct observation and through videotaping. 2. Chunking the data into discernible units of analysis that we have described as AREs. 3. Recording information related to the specifics that constitute each ARE (see Figure 1). 4. Developing a visual representation of the data by recording the time dura- tion of each ARE for each initiator, and then abstracting these times into numbered circles that are sequentially arranged in an ordinal, not ratio, fashion (see Figures 2 and 3). 5. Selecting the particular issue at hand, practice, or resource to serve as the tracer. 6. Tracing the historical development of the particular tracer over time by shading in all the related nodes, adding observed links, and examining the path (see Figure 4). 7. Reexamining node descriptions as well as videotapes to build interpretation of the network. If possible, performing member checks to validate interpre- tations. (p. 22)

INSTANTIATING THE CATEGORIES (OUR RESEARCH) (p. 23)

Research Context (p. 23)

Coding Examples (p. 24)

We have found it most efficient to enter these subcategories into a database program in which the items can be accessed as pop-up menus (see Figure 1 for the coding form de- veloped for our VSS context). In addition, it is essential that these items be editable and that items can be added as new subcategories are identified for is- sues at hand, practices, initiators, participants, and resources. (p. 24)

owever, in making the CN–ARE methodology a useful analytical tool, we have found it necessary to include ad- ditional fields to our form. (p. 24)

sort records (p. 24)

Conceptual richness is a rating on a 10-point scale ranging from 0 (nodes not related to course content)to9(nodes involving interconnections of ideas or systems-level understandings of the particular domain of interest). (p. 24)

Developing Codes (p. 25)

The codes related to our categories and subcategories emerged through the process of grounded theory development (p. 25)

Coding Scenarios as AREs (p. 25)

In these examples, we illustrate how we have parsed a scenario into separate AREs (nodes),3 and how we coded AREs. (p. 26)

To reiterate, AREs were delimited by a change in theme, activity, or initiator. In other words, whenever we observed a change in the issue at hand (e.g., from eclipses to animation), the practice (from modeling to Socratic questioning), or the initiator or participant (from one student to another), we coded a new node. (p. 26)

Interpreting Data (p. 32)

We have found it useful to carry out two types of data interpretations on the data- base of nodes generated through the CN–ARE methodology (Barab, Hay, Barnett, & Squire, in press): (a) as a database search tool to support frequency counts and grounded theory development (Glaser & Strauss, 1967), and (b) the N–ARE graphs discussed previously and further detailed later. (p. 32)

A database search tool. The database of coded interactions can simply be treated as a relational database, and thereby searched for various purposes. (p. 34)

Connecting nodes to build networks. A more innovative application of the coded information is for the generation of N–AREs. (p. 34)

we have found it useful to use the database to locate tracer-related nodes and then to graph them in an ordinal fashion. This network provides an inscription, a graphical representation, that can then be used by the researcher to scaffold her in- terpretation as well as her presentation of the data. (p. 34)

Focusing in on the practice of animation, they were able to document how the practice of animation, and the nested parent–child relations practice, diffused over the camp, frequently changing hands among students and involving independent ap- plication as well as more collaborative discussions. (p. 35)

Returning to the three example scenarios previously described, we now de- velop a N–ARE and then use it as an inscription to scaffold our interpretation of the data (see Figure 6). (p. 35)

The power of the CN–ARE methodology is that it captures and represents cognition in situ, showing how cognition is contextually embedded and distributed across concrete experiences. This approach allows us to move beyond capturing ready-made knowledge to capturing the situated dynamics that constitute know- ing in the-making (i.e., students actualizing conceptual tools in their model and as part of their collaborative dialogue). (p. 39)

TRUSTWORTHINESS, USEFULLNESS, AND LIMITATIONS (p. 39)

Is the Coding Scheme Trustworthy? (p. 39)

In spite of our ability to train a set of researchers who are then able to code segments with consistency, the trustworthiness of a coding scheme based on our subjective interpretations of such complex events is certainly not a straightfor- ward process. We are having to coordinate student gestures, dialogue, computer screens, and a class history all into a momentary judgement that occurs within the continuous flow of data. (p. 40)

Is the Coding Scheme Useful? (p. 40)

At one level, the use of the coding scheme to create a database of nodes is particularly useful for getting a broad look at an element, to search for particular episodes, for drawing con- trasts between groups (p. 40)

The more powerful function of the CN–ARE methodology is that it can be used to represent the historical development of cognition in situ, showing how cogni- tion is contextually embedded and distributed across concrete experiences. (p. 40)

What Are the Limitations? (p. 42)

Therefore, in order to actually capture these occurrences, it is necessary that researchers have large amounts of video data regarding student–student, stu- dent–teacher, student–tools, and student–resource interactions. Both capturing and analyzing this data are extremely labor intensive, and, in many situations, simply an impractical task (see Jordan & Henderson, 1995). Therefore, this approach is ap- propriate for researchers but will most likely have little application, at this stage, to the classroom teacher—although interpretations derived from analysis can prove useful to teachers. (p. 42)

where more automated process would help (now) (p. 42)

Other limitations include the time-consuming process of training coders, getting them properly situated in each new context, and the qualitative nature of determining the boundaries of a particular node. (p. 42)

There have been numerous critiques targeted at network theory more generally. For example, in critique against the Actor Network Theory (ANT) approach, Engeström and Escalante (1996) stated: “In its search for convergence, irreversibilization, and closure, this kind of analysis overlooks the inner dynamics and contradictions of the activities of the various actors in the network” (p. 344). They further stated that “the concepts of trust and reciprocity, so central in new theo- rizing on network organizations, and the whole contradictory dialectic of coopera- tion and competition, are curiously missing in the vocabulary of actor-network theory” (p. 46). (p. 43)

Clearly psychological, cultural, and social factors are an important part of understanding situations when one adopts a situated perspective on what it means to know and learn. (p. 43)

We have found that coupling the network story with a more general ethnographic account provides a much richer description of the context in gen- eral. (p. 43)

IMPLICATIONS (p. 46)

One potential expansion of this methodology is to add quantitative information to the information coded at the node level—in a sense, quantifying the qualitative analysis of the data. (p. 47)

We are at a time of paradigmatic shifts in ontology, epistemology, and peda- gogy, and researchers need to continue to look for novel techniques that are able to capture cognition conceived as situated. (p. 47)

However, commensurate with recent epistemological shifts, it is a time for real and meaningful exploration, applying data analytic techniques that afford researchers rich descriptions of the process through which learners become knowledgeably skillful within the context of their participation. (p. 47)

Barab, S. A., Hay, K. E., Barnett, M. G., & Squire, K. (in press). Constructing virtual worlds: Tracing the historical development of learner practices. Cognition and Instruction. (p. 48)