read

References

Citekey: @Breslow2013

Breslow, L. and Pritchard, D. (2013). Studying learning in the worldwide classroom: Research into edX’s first MOOC. Research & Practice Assessment, 8(1):13–25.

Notes

This article is one of my favorite piece about MOOC research. It was also one of the earliest to look more deeply into MOOC learner activities. The questions got answered are important, and preliminary at the same time. But these many questions the authors ask are even more important to further explore. Obviously the team has a lot to offer in the coming year or so to help us better understand MOOCs.

Highlights

“Circuits and Electronics” (6.002x), which began in March 2012, was the first MOOC developed by edX (p. 13)

This article describes both the first stage of this research, which examined the students’ use of resources by time spent on each, and a second stage that is producing an in-depth picture of who the 6.002x students were, how their own background and capabilities related to their achievement and persistence, and how their interactions with 6.002x’s curricular and pedagogical components contributed to their level of success in the course. (p. 13)

This paper describes an initial study of the data generated by MIT’s first MOOC, “Circuits and Electronics” (6.002x)1 by a team of multidisciplinary researchers from MIT and Harvard. These data include the IP addresses of all enrolled students; clickstream data that recorded each of the 230 million interactions the students had with the platform (Seaton, Bergner, Chuang, Mitros, & Pritchard, 2013); scores on homework assignments, labs, and exams; student and teaching staff posts on a discussion forum; and the results of a survey sent to the 6.002x students at the end of the course. We are trying to understand who the students were in 6.002x, how they utilized course resources, what contributed to their persistence, and what advanced or hindered their achievement. (p. 14)

If educational researchers studying conventional brick and mortar classrooms struggle to operationalize variables like attrition and achievement, it is doubly difficult to do so for MOOCs. (p. 14)

Participation and performance do not follow the rules by which universities have traditionally organized the teaching enterprise: MOOCs allow free and easy registration, do not require formal withdrawals, and include a large number of students who may not have any interest in completing assignments and assessments. (p. 14)

First Study Explores Resource Usage (p. 15)

Plot A highlights the weekly periodicity; peaks on weekends presumably reflect both the days when spare time is available and the deadline for homework submission. In plots B and C activity is shown in hits per user each day. The three instructional resources—textbook, video lectures, and lecture questions—display little end-of-week peaking, whereas for–credit assessments (homework and labs) show marked peaks suggesting these activities were done just ahead of the deadline. The discussion forum shows similar periodicity because it is accessed while doing the homework problems (for more on the use of the discussion forum, please see below). The drop in e-text activity after the first exam is typical of textbook use that has been observed in blended on-campus courses where the textbook was a supplementary resource (that is, not part of the sequence of activities presented to students by the interface). (p. 16)

Time represents the principal cost function for students, and it is therefore important to study how students allocated their time throughout the course. (p. 16)

Clearly, the most time was spent on lecture videos (see Figure 3). (p. 16)

Anotherinterestingfeaturerevealedbythesedataisstudentstrategyinsolvingproblems. By strategy, we mean which resources were most frequently consulted by the students while doing problems, and which ones were viewed for the longest time? Student strategy differs very markedly when solving homework problems versus when solving exam problems. (p. 17)

Second Stage of Research Examines Demographics, Achievement, and Persistence (p. 17)

“Who were the students who enrolled in 6.002x, and what factors related to their level of success in the course?” (p. 17)

attempting to construct a detailed picture of the 6.002x students, using multiple sampling frames: all registrants, all students who clicked on the course website, students who demonstrated different levels of engagement of the course, and certificate earners. Next, we hope to be able to identify relationships between the characteristics and capabilities of the students themselves and their success. Finally, we want to understand how the curricular and pedagogical components of 6.002x contributed to the students’ ability to master the material. (p. 17)

Diversity in Location and Demographics (p. 17)

Students came from 194 countries, virtually all in the world. (p. 17)

We know, too, from an open-ended profile edX posted at the start of the course, 67% of registrants spoke English, and 16% , the next largest group, spoke Spanish. (p. 18)

The survey questions, which were grounded in research in large-scale studies in international education, included not only demographics such as age and gender, but asked students, for example, about their home environment and their educational and professional background. This is in line with educational research (Coleman et al., 1966; Gamoran & Long, 2008) that indicates these latter variables serve as important controls in predictions of educational outcomes. (p. 18)

most reported they were in their 20s and 30s (p. 18)

As might also be predicted, 88% of those who reported their gender were male. (p. 19)

not surprised to learn that over half the survey respondents reported the primary reason they enrolled in 6.002x was for the knowledge and skills they would gain. Although, interestingly, only 8.8% stated they registered for the course for “employment or job advancement opportunities.” Over a quarter of the students took the course for the “personal challenge.” (p. 19)

There were no correlations between motivation for enrollment and success in the course. Whether students were taking 6.002x to advance their knowledge or because they wanted the challenge (we realize, of course, the two could be interrelated), it did not seem to affect their performance in the class. (p. 19)

Same results
We found the same in the MRI project (p. 19)

What Contributed to Student “Success”? Predictive Modeling as the Next Step in the Analysis (p. 19)

The first stage in this work is to define more precisely what we mean by “success” in a MOOC. (p. 19)

Success as Achievement (p. 20)

Thus, we argue, that “success” in 6.002x can be defined as it is labs, and a midterm and final. Thus, we argue, that “success” in 6.002x can be defined as it is in the traditional college classroom, namely, by the grades students earned. (p. 20)

Using this definition, we found no relationship between age and achievement or Using this definition, we found no relationship between age and achievement or “success” in 6.002x can “success” in 6.002x can between gender and achievement, and we found only a marginal relationship between between gender and achievement, and we found only a marginal relationship between be defined as it is in the be defined as it is in the highest degree earned and achievement. (p. 20)

The strongest correlation we found between what we are calling “student background” The strongest correlation we found between what we are calling “student background” and achievement was in whether or not the survey respondent “worked offline with anyone and achievement was in whether or not the survey respondent “worked offline with anyone on the MITx material.” (p. 20)

On average, with area,” as 2.5% did, that interaction seemed to have had a beneficial effect. On average, with all other predictors being equal, a student who worked offline with someone else in the class all other predictors being equal, a student who worked offline with someone else in the class or someone who had expertise in the subject would have a predicted score almost three or someone who had expertise in the subject would have a predicted score almost three points higher than someone working by him or herself. (p. 20)

For some of these analyses, we have experimented with operationalizing “achievement” in two different ways: as scores on homework assignments or performance on the final. (p. 21)

Success as Persistence (p. 21)

One of the more troubling aspects of MOOCs to date is their low completion rate, which averages no more than 10%. This was true of 6.002x as well, with less than 5% of the students who signed up at any one time completing the course. (p. 21)

We want to understand more about stop out, so we are also operationalizing “success” as “persistence throughout the duration of the course.” Here, too, we are working with multiple possible definitions: persistence can be “interaction with any part of the course in any subsequent week” or “interaction with a specific course component in any subsequent week.” (p. 22)

We are then estimating a survival function based on student use of resources. While the use of some resources seems to predict an increased likelihood of stopping out of the class in the next week, interactions with other resources seem to predict a decrease in likelihood of stop out. We are extending this model to look at time-varying risk functions—factors that might increase the likelihood of stopping out at the beginning of the course but have the opposite effect at the end of the course. (p. 22)

Research on the Discussion Forum and On-Campus Use of 6.002x (p. 22)

Over 12,000 discussion threads were initiated during 6.002x, including almost 100,000 individual posts, providing a rich sample for this analysis. (p. 22)

However, we know that, on average, only 3% of all students participated in the discussion forum. (p. 22)

In total, 52% of the certificate earners were active on the forum. (p. 22)

Our initial approach in exploring the discussion forum has been to categorize these interactions very broadly along two dimensions: (a) topic (i.e., course content, course structure or policies, course website or technology, social/affective), and (b) role of the student posting (i.e., help-seeker/ information-seeker or help-giver/information-giver). (p. 23)

we will be able to describe the general purposes for which the forum was used (p. 23)

what is the nature of the interactions that create a productive collaboration? (p. 23)

We want to understand how “discussion” might have helped 6.002x students to unravel a misconception, understand a difficult topic, or employ an algorithmic procedure. To do this, we are looking more specifically at threads in which students sought and received help on complex homework problems. (p. 23)

We are examining the quantity of interactivity between question askers and responders, as well as inferences made by both parties. As yet another means of exploring these data, we are experimenting with social network analysis to see if it yields findings about the nature and longevity of group formation in 6.002x. (p. 23)

The appearance of MOOCs in higher education has been swift—so swift, in fact, that it could be called unprecedented. Since their introduction only a scant 18 months ago, there has been no shortage of prophecies about their potential impact. Those predictions have run the gamut from the wildly hopeful to the bleakly dire. The optimists see MOOCs expanding access to previously disenfranchised groups of students, developing new methods of pedagogy for deeper, more sustained learning, and building global communities focused not on the latest fad or celebrity, but on education. Doomsayers predict the end of liberal learning, a generation unable to communicate in face-to-face classrooms, and even the eventual demise of the university. What the two camps agree on—and what history and current events indicate— is that it is unlikely that higher educational will not be affected by MOOCs. (p. 23)

References

Hart, C. (2012). Factors associated with student persistence in an online program of study: A review of the literature. Journal of Interactive Online Learning, 11(1), 19-42. (p. 25)

Seaton, D. T., Bergner, Y., Chuang, I., Mitros, P., & Pritchard, D. E. (2013). Who does what in a massive open online course? Communications of the ACM.

Blog Logo

Bodong Chen


Published

Image

Crisscross Landscapes

Bodong Chen, University of Minnesota

Back to Home