read

References

Citekey: @slade2013learning

Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American Behavioral Scientist, 57(10), 1510–1529. doi:10.1177/0002764213479366

Notes

Highlights

This article proposes a sociocritical perspective on the use of learning analytics. Such an approach highlights the role of power, the impact of surveillance, the need for transparency, and an acknowledgment that student identity is a transient, temporal, and context-bound construct. Each of these affects the scope and definition of learning analytics’ ethical use. We propose six principles as a framework for considerations to guide higher education institutions to address ethical issues in learning analytics and challenges in context-dependent and appropriate ways. (p. 1510)

In the field of learning analytics, discussions around the ethical implications of increasing an institution’s scrutiny of student data typically relate to ownership of that data and to student privacy issues. (p. 1511)

Approaches taken to understand the opportunities and ethical challenges of learning analytics necessarily depend on a range of ideological assumptions and epistemologies. For example, if we approach learning analytics from the perspective of resource optimization in the context of the commoditization of higher education (Hall & Stahl, 2012), the ethical issues appear different from those resulting from a sociocritical perspective. In addition, learning analytics has evolved from a range of research areas such as social network analysis, latent semantic analysis, and dispositions analysis (Ferguson, 2012). Each of these domains has its own, often overlapping, ethical guidelines and codes of conduct that address similar concerns such as the ownership of data, privacy, consumer or patient consent, and so on. (p. 1511)

For this article, we situate learning analytics within an understanding of power relations among learners, higher education institutions, and other stakeholders (e.g., regulatory and funding frameworks). Such power relations can be considered from the perspective of Foucault’s Panopticon, where structural design allows a central authority to oversee all activity. In the case of learning analytics, such oversight or surveillance is often available only to the institution, course designers, and faculty, and not to the student (Land & Bayne, 2005). (p. 1511)

A sociocritical perspective entails being critically aware of the way our cultural, political, social, physical, and economic contexts and power relationships shape our responses to the ethical dilemmas and issues in learning analytics (see, e.g., Apple, 2004). Ethical issues for learning analytics fall into the following broad, often overlapping categories: 1. The location and interpretation of data 2. Informed consent, privacy, and the deidentification of data 3. The management, classification, and storage of data (p. 1511)

Such ethical issues are not unique to education, and similar debates relating to the use of data to inform decisions and interventions are also found in the health sector (Cooper, 2009; Dolan, 2008; Snaedal, 2002), human resource management (Cokins, 2009), talent management (Davenport, Harris, & Shapiro, 2010), homeland security (Seifert, 2004), and biopolitics (Dillon & Loboguerrero, 2008). Of specific concern here are the implications of viewing learning analytics as moral practice (p. 1511)

Perspectives on Ethical Issues in Learning Analytics (p. 1512)

Oblinger (2012) differentiates between learning analytics and approaches including business intelligence and academic analytics, defining learning analytics as focusing on “students and their learning behaviors, gathering data from course management and student information systems in order to improve student success” (p. 11). (p. 1512)

An Overview of the Purposes and Collection of Educational Data (p. 1512)

Ethical challenges and issues in the use of educational data can be usefully viewed in the context of the history of Internet research ethics (p. 1512)

As a result, there are a growing number of ethical issues regarding the collection and analyses of educational data, issues that include greater understanding and transparency regarding the “purposes for which data is being collected and how sensitive data will be handled” (Oblinger, 2012, p. 12). Legal frameworks such as the U.S. Family Educational Rights and Privacy Act focus largely on how information is used outside an institution rather than on its use within the institution, and Oblinger (2012) argues that it is crucial that institutions inform students “what information about them will be used for what purposes, by whom, and for what benefit” (p. 12). (p. 1513)

Subotzky and Prinsloo (2011) suggest that there is a need for a reciprocal sharing of appropriate and actionable knowledge between students and the delivering institution. Such knowledge of students may facilitate the offering of just-in-time and customized support, allowing students to make more informed choices and act accordingly. (p. 1513)

The Educational Purpose of Learning Analytics (p. 1513)

higher education “has traditionally been inefficient in its data use, often operating with substantial delays in analyzing readily evident data and feedback” (Long & Siemens, 2011, p. 32). (p. 1513)

Booth (2012) emphasizes that learning analytics also has the potential to serve learning. A learning analytics approach may make education both personal and relevant and allow students to retain their own identities within the bigger system. Optimal use of student-generated data may result in institutions having an improved comprehension of the lifeworlds and choices of students, allowing both institution and students to make better and informed choices and respond faster to actionable and identified needs (Oblinger, 2012). (p. 1513)

Several authors (Bienkowski, Feng, & Means, 2012; Campbell, DeBlois, & Oblinger, 2007) refer to the obligation that institutions have to act on knowledge gained through analytics. (p. 1513)

Amid the emphasis on the role of data and analyses for reporting on student success, retention, and throughput, it is crucial to remember that learning analytics has huge potential to primarily serve learning (Kruse & Pongsajapan, 2012). When institutions emphasize the analysis and use of data primarily for reporting purposes, there is a danger of seeing students as passive producers of data, resulting in learning analytics used as “intrusive advising” (Kruse & Pongsajapan, 2012, p. 2). (p. 1514)

Power and Surveillance in Learning Analytics (p. 1514)

Owing to the inherently unequal power relations in the use of data generated by students in their learning journey, we propose a sociocritical framework that allows us to address a range of ethical questions such as levels of visibility, aggregation, and surveillance. (p. 1514)

Dawson (2006), for example, found that students altered their online behaviors (e.g., range of topics discussed and writing style) when aware of institutional surveillance. Albrechtslund (2008, para. 46) explores the notion of participatory surveillance, focusing on surveillance as “mutual, horizontal practice” as well as the social and “playful aspects” of surveillance (also see Knox, 2010b; Lyon, 2007; Varvel, Montague, & Estabrook, 2007). Knox (2010b) provides a very useful typology of surveillance, highlighting, inter alia, the difference between surveillance (p. 1514)

and monitoring, automation and visibility, and various aspects of rhizomatic and predictive surveillance. Rhizomatic surveillance highlights the dynamic, multidirectional flow of the act of surveillance in a synopticon, where the many can watch the few. The synopticon and Panopticon function concurrently and interact (Knox, 2010b). (p. 1515)

The following sections discuss a range of ethical issues grouped within three broad, overlapping categories: 1. The location and interpretation of data 2. Informed consent, privacy, and the deidentification of data 3. The management, classification, and storage of data (p. 1515)

  1. The Location and Interpretation of Data (p. 1515)

“significant amounts of learner activity take place externally [to the institution] . . . records are distributed across a variety of different sites with different standards, owners and levels of access” (Ferguson, 2012, para. 6). (p. 1515)

Not only do we not have all the data, a lot of the data that we do have require “extensive filtering to transform the ‘data exhaust’ in the LMS log file into educationally relevant information” (Whitmer et al., 2012, para. 22). (p. 1515)

There are implications, too, of ineffective and misdirected interventions resulting from faulty learning diagnoses that might result in “inefficiency, resentment, and demotivation” (Kruse & Pongsajapan, 2012, p. 3). (p. 1515)

Given the wide range of information that may be included in such models, there is a recognized danger of potential bias and oversimplification (Bienkowski et al., 2012; Campbell et al., 2007; May, 2011). In accepting the inevitability of this, should we also question the rights of the student to remain an individual and whether it is appropriate for students to have an awareness of the labels attached to them? Are there some labels that should be prohibited? (p. 1516)

  1. Informed Consent, Privacy, and the Deidentification of Data (p. 1516)

el and Royakkers (2004) discuss the ethics of tracking and analyzing student data without their explicit knowledge. Of interest, Land and Bayne (2005) discuss the broad acceptance of student surveillance and cite studies in which they record that the concept of logging educational activities seems to be quite acceptable to students. The notion of online privacy as a social norm is increasingly questioned (Arrington, 2010; Coll, Glassey, & Balleys, 2011). (p. 1516)

Petersen (2012) points to the importance of the deidentification of data before the data are made available for institutional use, including the option to “retain unique identifiers for individuals in the data set, without identifying the actual identity of the individuals” (p. 48). (p. 1516)

  1. The Classification and Management of Data (p. 1516)

Petersen (2012) proposes a holistic approach to transparent data management, including a need for a “comprehensive data-governance structure to address all the types of (p. 1516)

data used in various situations,” addressing the need to create “a data-classification system that sorts data into categories that identify the necessary level of protection” (pp. 46-47). (p. 1517)

With regard to the importance of trust in monitoring and surveillance (e.g., Adams & Van Manen, 2006; Knox, 2010a; Livingstone, 2005), we agree that the classification of data, as proposed by Petersen (2012), is an essential element in ensuring that appropriate access to different types of data will be regulated. (p. 1517)

Integral to contemplating the ethical challenges in learning analytics is a consideration of the impact of the tools used. (p. 1517)

as highlighted by Pariser (2011), that pattern recognition can result in keeping individuals prisoner to past choices. Pariser suggests that the use of personalized filters hints of “autopropaganda, indoctrinating us with our own ideas, amplifying our desire for things that are familiar,” and that “knowing what kinds of appeals specific people respond to gives you the power to manipulate them on an individual basis” (p. 121). (p. 1517)

The dynamic nature of student identity necessitates that we take reasonable care to allow students to act outside of imposed algorithms and models. (p. 1517)

Student Identity as Transient Construct (p. 1517)

it is crucial that the analysis of data in learning analytics keeps in mind the temporality of harvested data and the fact that harvested data allow us only a view of a person at a particular time and place. (p. 1517)

Institutions should also recognize the plurality of student identity. Sen (2006) suggests that we should recognize identities as “robustly plural, and that the importance of one identity need not obliterate the importance of others” (p. 19). Students, as agents, make choices—“explicitly or by implication—about what relative importance to attach, in a particular context, to the divergent loyalties and priorities that may compete for precedence” (p. 19; also see Brah, 1996). (p. 1517)

Toward an Ethical Framework (p. 1518)

The review left us with this question: How do we address both the potential of learning analytics to serve learning and the associated ethical challenges? (p. 1518)

Petersen (2012, p. 48) proposes adherence to the principles found in the U.S. Federal Trade Commission’s Fair Information Practice Principles, which cover the elements of informed consent, allowing different options regarding the use of data, individuals’ right to check the accuracy and completeness of information, preventing unauthorized access, use, and disclosure of data, and provisions for enforcement and redress. Buchanan (2011) proposes three ethical principles, namely “respect for persons, beneficence, and justice” (p. 84). Not only should individuals be regarded as autonomous agents, but vulnerable individuals with diminished or constrained autonomy (including students) should be protected from harm and risk. (p. 1518)

Principles for an Ethical Framework for Learning Analytics (p. 1518)

Our approach holds that an institution’s use of learning analytics is going to be based on its understanding of the scope, role, and boundaries of learning analytics and a set of moral beliefs founded on the respective regulatory and legal, cultural, geopolitical, and socioeconomic contexts. (p. 1518)

As such, it would be almost impossible to develop a set of universally valid guidelines that could be equally applicable within any context. However, it should be possible to develop a set of general principles from which institutions can develop their own sets of guidelines depending on their contexts. (p. 1519)

Principle 1: Learning Analytics as Moral Practice (p. 1519)

learning analytics should do much more than contribute to a “datadriven university” or lead to a world where we are “living under the sword of data.” We agree with Biesta (2007) that evidence-based education seems to favor a technocratic model in which it is assumed that the only relevant research questions are about the effectiveness of educational means and techniques, forgetting, among other things, that what counts as “effective” crucially depends on judgments about what is educationally desirable. (p. 5) (p. 1519)

Education cannot and should not be understood as “as an intervention or treatment because of the noncausal and normative nature of educational practice and because of the fact that the means and ends in education are internally related” (Biesta, 2007, p. 20). Learning analytics should not only focus on what is effective, but also aim to provide relevant pointers to decide what is appropriate and morally necessary. Education is primarily a moral practice, not a causal one. Therefore, learning analytics should function primarily as a moral practice resulting in understanding rather than measuring (Reeves, 2011). (p. 1519)

Principle 2: Students as Agents (p. 1519)

In stark contrast to seeing students as producers and sources of data, learning analytics should engage students as collaborators and not as mere recipients of interventions and services (Buchanan, 2011; Kruse & Pongsajapan, 2012). (p. 1519)

they should also voluntarily collaborate in providing data and access to data to allow learning analytics to serve their learning and development, and not just the efficiency of institutional profiling and interventions (also see Subotzky & Prinsloo, 2011). Kruse and Pongsajapan (2012) propose a “student-centric,” as opposed to an “intervention-centric,” approach to learning analytics. This suggests the student should be seen as a co-interpreter of his own data—and perhaps even as a participant in the identification and gathering of that data. In this scenario, the student becomes aware of his own actions in the system and uses that data to reflect on and potentially alter his behavior. (pp. 4-5) (p. 1519)

Valuing students as agents, making choices and collaborating with the institution in constructing their identities (however transient), can furthermore be a useful (and powerful) antidote to the commercialization of higher education (see, e.g., Giroux, 2003) in the context of the impact of skewed power relations, monitoring, and surveillance (Albrechtslund, 2008; Knox, 2010a, 2010b). (p. 1520)

Principle 3: Student Identity and Performance Are Temporal Dynamic Constructs (p. 1520)

It is crucial to see student identity as a combination of permanent and dynamic attributes. (p. 1520)

Mayer-Schönberger (2009, p. 12) warns that forgetting is a “fundamental human capacity.” Students should be allowed to evolve and adjust and learn from past experiences without those experiences, because of their digital nature, becoming permanent blemishes on their development history. Student profiles should not become “etched like a tattoo into . . . [their] digital skins” (Mayer-Schönberger, 2009, p. 14). (p. 1520)

Principle 4: Student Success Is a Complex and Multidimensional Phenomenon (p. 1520)

it is important to see student success is the result of “mostly non-linear, multidimensional, interdependent interactions at different phases in the nexus between student, institution and broader societal factors” (Prinsloo, 2012). Although learning analytics offer huge opportunities to gain a more comprehensive understanding of student learning, our data are incomplete (e.g., Booth, 2012; Mayer-Schönberger, 2009; Richardson, 2012a, 2012b) and “dirty” (Whitmer et al., 2012) and our analyses vulnerable to misinterpretation and bias (Bienkowski et al., 2012; Campbell et al., 2007; May, 2011). (p. 1520)

Principle 5: Transparency (p. 1520)

Important for learning analytics as moral practice is that higher education institutions should be transparent regarding the purposes for which data will be used and under which conditions, who will have access to data, and the measures through which individuals’ identity will be protected. (p. 1520)

Principle 6: Higher Education Cannot Afford to Not Use Data (p. 1521)

The sixth principle makes it clear that higher education institutions cannot afford to not use learning analytics. (p. 1521)

Ignoring information that might actively help to pursue an institution’s goals seems shortsighted to the extreme. Institutions are accountable, whether it is to shareholders, to governments, or to students themselves. Learning analytics allows higher education institutions to assist all stakeholders to penetrate “the fog that has settled over much of higher education” (Long & Siemens, 2011, p. 40). (p. 1521)

Considerations for Learning Analytics as Moral Practice (p. 1521)

we propose a number of considerations rather than a code of practice to allow for flexibility and the range of contexts in which they might need to be applied. (p. 1521)

Who Benefits and Under What Conditions? (p. 1521)

The answer to this question is the basis for considering the ethical dimensions of learning analytics. (p. 1521)

Students are not simply recipients of services or customers paying for an education. They are and should be active agents in determining the scope and purpose of data harvested from them and under what conditions (e.g., deidentification). (p. 1521)

Conditions for Consent, Deidentification, and Opting Out (p. 1522)

Given that the scope and nature of available data have changed dramatically, we should revisit the notion of informed consent in the field of learning analytics. (p. 1522)

There are many examples in different fields (e.g., bioethics) where the principle of informed consent can be waived under predetermined circumstances or if existing legislation is sufficient (e.g., data protection legislation). In extreme cases, informed consent may be forgone if the benefit to the many exceeds the needs of the individual. (p. 1522)

In the context of learning analytics, we might suggest that there are few, if any, reasons not to provide students with information regarding the uses to which their data might be put, as well as the models used (as far as they may be known at that time), and to establish a system of informed consent. (p. 1522)

this consent may need to be refreshed on a regular basis. (p. 1522)

She suggests the need to achieve a reasonable balance between allowing quality research to be conducted and protecting users from potential harm. (p. 1522)

Buchanan (2011), referring to the work of Lawson (2004), suggests a nuanced approach to consent that offers students a range of options for withholding (partial) identification of individuals where they are part of a published study. In light of this, it seems reasonable to distinguish between analyzing and using anonymized data for reporting purposes to regulatory bodies or funding purposes and other work on specific aspects of student engagement. In the context of reporting purposes, we support the notion that the benefit for the majority supersedes the right of the individual to withhold permission for use of his or her data. Students may, however, choose to opt out of institutional initiatives to personalize learning—on the condition that students are informed and aware of the consequences of their decision. (p. 1522)

Institutions should also provide guarantees that student data will be permanently deidentified after a certain number of years, depending on national legislation and regulatory frameworks. (p. 1522)

Vulnerability and Harm (p. 1522)

We suggest that the potential for bias and stereotyping in predictive analysis should be foregrounded in institutional attempts to categorize students’ risk profiles. Institutions should provide additional opportunities for these students either to prove the initial predictive analyses wrong or incomplete or to redeem themselves despite any initial institutional doubt regarding their potential. (p. 1523)

Data Collection, Analyses, Access, and Storage (p. 1523)

It is generally accepted that data on the institutional LMS provide an incomplete picture of learners’ learning journeys. As learners’ digital networks increasingly include sources outside of the LMS, institutions may utilize data from outside the LMS (e.g., Twitter and Facebook accounts, whether study related or personal) to get more comprehensive pictures of students’ learning trajectories. (p. 1524)

Institutions should commit themselves to take due care to prevent bias and stereotyping, always acknowledging the incomplete and dynamic nature of individual identity and experiences. Algorithms used in learning analytics inherently may reflect and perpetuate the biases and prejudices in cultural, geopolitical, economic, and societal realities. (p. 1524)

Care should be taken to ensure that the contexts of data analyzed are carefully considered before tentative predictions and analyses are made. (p. 1524)

In line with Kruse and Pongsajapan’s (2012) proposal for “studentcentric” learning analytics, we propose that students have a right to be assured that their data will be protected against unauthorized access and that their informed consent (as discussed above) is guaranteed when their data are used. (p. 1524)

Institutions should provide guarantees and guidelines with regard to the preservation and storage of data in line with national and international regulatory and legislative frameworks. Students should be informed that their data will be encrypted. (p. 1525)

Governance and Resource Allocation (p. 1525)

Petersen (2012, P. 46) states, “The most important step that any campus can take is to create a comprehensive data-governance structure to address all the types of data used in various situations.” (p. 1525)

Conclusions (p. 1525)

Learning analytics is primarily a moral and educational practice, serving better and more successful learning. The inherent peril and promise of having access to and analyzing “big data” (Bollier, 2010) necessitate a careful consideration of the ethical dimensions and challenges of learning analytics. The proposed principles and considerations included within this article provide an ethical framework for higher education institutions to offer context-appropriate solutions and strategies to increase the quality and effectiveness of teaching and learning. (p. 1526)

Blog Logo

Bodong Chen


Published

Image

Crisscross Landscapes

Bodong Chen, University of Minnesota

Back to Home