Bodong Chen

Crisscross Landscapes

Notes: Thompson. (2013). Processing and Visualizing Data in Complex Learning Environments



Citekey: @Thompson2013

Thompson, K., Ashe, D., Carvalho, L., Goodyear, P., Kelly, N., & Parisio, M. (2013). Processing and Visualizing Data in Complex Learning Environments. American Behavioral Scientist, 57(10), 1401–1420. doi:10.11770002764213479368


I like the proposed framework and model. But the presence of LA in Case 1 is especially minimal. Case 2 involves much human coding. Appears to be a more traditional CSCL research.


We discuss, using the cases of two learning environments, how structure affects the behavior of learners and, in turn, how that behavior has the potential to affect learning. (p. 1401)

Learning analytics has focused on making sense of “big data,” data usually collected from learning management systems (e.g., Macfadyen & Dawson, 2010). The automatic extraction of data is then used for analysis at the course and student levels (e.g., social network analysis; Haythornthwaite & de Laat, 2011). Often, learning analytics focuses on the aim of identifying highor low-achieving students so that an intervention can be made effectively (e.g., Dawson, Macfadyen, & Lockyer, 2009). (p. 1401)

Wondering how many would agree with these statements. (p. 1401)

We describe our experiences processing and visualizing data in two complex learning environments. The first is an informal online learning environment used in Open University courses, open to anyone. The second is a small group of master’s students collaborating on a task. Throughout this article, the analytical relationship that we draw attention to is between the structure of a learning environment (virtual or physical; the “designable component”) and the learners’ behavior. For each case study, we discuss the selection of the elements and information this provides about the learners. We then discuss ways to apply other learning analytics techniques to these case studies. (p. 1402)

Digital technology, as with other objects in the broader material world, is not neutral and has ways of influencing human perception and action. (p. 1402)

The links between physical properties, perception, and action involve human processes, such as thinking and interpretation, that require varying degrees of conscious attention, mental effort, or cognitive load (Kahneman, 2011). (p. 1403)

We have developed an analytical framework to identify and represent key elements of complex learning environments, which allows us to explore the nature of the physical, digital, and human elements within learning environments; what and how tools, texts, and artifacts are produced and/or manipulated; and the types of tasks, rules, and divisions of labor that become established within a range of learning contexts (Carvalho & Goodyear, in press). The framework establishes four analytical dimensions. Three of these reflect the major design components, which we label set design (the physical-digital “stage” on which learning activity is situated; the tools, artifacts, and so on that come to hand); epistemic design (tasks proposed, knowledge implicated); and social design (such as roles, divisions of labor). Since network participants’ activities also result in rearrangements of the learning environment, we use a fourth analytic dimension to capture co-creation and co-configuration. (p. 1403)

This analytical framework offers ways of identifying and representing key elements of complex learning environments. (p. 1403)

The researcher needs to see the big picture and to gain an overview of how the individual data elements interact and relate. (p. 1405)

The focus in both cases discussed below is on the demonstration of expertise in individual learners as an indicator of successful collaboration. (p. 1405)

Case Study 1: iSpot (p. 1406)

The data collected and analyzed comprised screenshots of the environment, featuring the design elements we were interested in: the butterfly icons in iSpot (Figure 2). The analysis of design elements was underpinned by concepts from semiotics and design. Nadin (1988) refers to design principles as being semiotic by nature (p. 1407)

to design means to structure a system of signs in such a way as to make possible the achievement of human goals: communication (as a form of social interaction), engineering (as a form of applied technical rationality), business (as a form of shared efficiency), architecture, art, education, etc. (p. 269) (p. 1408)

Complex learning environments require analytical tools to help people understand how design elements may influence or shape learning within a particular context. (p. 1408)

Case Study 2: Face-to-Face Collaboration (p. 1409)

Our second case study involves technology-enhanced collaboration. The aim is to identify indicators of expertise used in a face-to-face learning environment, adapting learning analytics techniques to do so. (p. 1409)

A corpus of student utterances available in a transcript from the audio files was used for the analysis. (p. 1409)

We coded the video data according to tool use and object of attention for each second (p. 1412)

Discussion (p. 1415)

In the two case studies that we present here, notions from learning analytics are applied. The iSpot case study makes use of the data available from a community website to analyze the effects of a design feature of the learning environment, namely, the role of the icons (set) in communicating expertise and the way in which this changes over time. The second case study of face-to-face learning uses both physical and verbal data from interaction to analyze learning processes. In both cases, there is a set of data that demonstrates how to apply analytics techniques. Many techniques are applicable and it is through the application of analysis within a framework to answer a question that specific methods are selected. (p. 1415)