Bodong Chen

Crisscross Landscapes

Notes: Liyanagunawardena-2013-A Systematic Study of the Published Literature 2008-2012



Citekey: @Liyanagunawardena2013-ku

Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: A Systematic Study of the Published Literature 2008-2012. The International Review of Research in Open and Distributed Learning, 14(3), 202–227.


Summarize: An early review of the MOOC literature. Forty-five peer reviewed papers are identified, using several different search strategies. Several glitches/hurdles were noted, such as other research using the same acronym. Quantitative analysis deals with pub year, counts in journals, leading authors, etc. Qualitative content analysis arrives at eight different areas of interest, introductory, concept, case studies, educational theory, technology, participant focussed, provider focussed, and other.

Assess: A useful study of early stage development of MOOC research. As expected, most results reflect cMOOCs (esp case studies of cMOOCs). Worth considering together with later studies.

Reflect: NA


This paper presents a systematic review of the published MOOC literature (2008-2012): Forty-five peer reviewed papers are identified through journals, database searches, searching the Web, and chaining from known sources to form the base for this review. (p. 1)

The review categorises the literature into eight different areas of interest, introductory, concept, case studies, e ducational theory, technology, participant focussed, provider focussed, and other, while also providing quantitative analysis of publications according to publication type, year of publication, and contributors. (p. 1)

Method (p. 4)

Data Collection (p. 4)

Relevant papers were identified through a series of search efforts, using an approach based on the methods used in other systematic reviews including two studies of literature related to the microblogging system Twitter (Gao, Luo, & Zhang, 2012; Williams, Terras, & Warwick, 2013). Papers were classed as relevant if their primary focus was to explore the concept of a MOOC or the implications for higher education, report on experiments with MOOCs, or compare MOOCs with other educational approaches. There was insufficient statistical data across the papers found to undertake a meta-analysis (Fink 2010). (p. 4)

Firstly, the search terms and boundaries to be used were established. Initially two search terms (and their plurals) were selected: • MOOC • Massively Open Online Course However it was identified that some authors used “Massive” instead of “Massively” (for example Kop, Fournier, & Mak 2011) and so the third term (and its plural) was added: • Massive Open Online Course (p. 4)

The search period was limited to the period from the year in which the first MOOC was run (2008) to the year this study started (2012). (p. 4)

Secondly, following the approach of Gao, et al. ( 2012) , a number of academic journals in the disciplines of educational technology and distance education were selected for a preliminary search (Table 1). These journals were The British Journal of Educational Technology, Distance Education, American Journal of Distance Education, and Journal of Online Learning and Teaching. (p. 4)

Thirdly, the same search terms were then used to search various academic databases: ISI Web of Knowledge, ProQuest (ERIC, British Education Index and Australian Education Index), JSTOR (education titles), IEEEXplorer, and Scopus (Table 2). A number of the articles returned were unrelated to this study, for example papers returned included some on “Multiple Optical Orthogonal Code Sequences” and some on “Management of Organizational Change”2, both of which were abbreviated to MOOC. (p. 5)

important to note (p. 5)

Fourthly, the same search was conducted using the Google Scholar search engine with 94 results (November 21, 2012). From these results only 33 were found to be relevant. (p. 6)

In order to improve the coverage of relevant publications to be included in this review, the chaining technique of Gao, et al. (2012) was used: consulting the reference lists of papers that were already in the corpus to locate other relevant work. (p. 6)

The data collection process resulted in the identification of 45 distinct articles – 17 from journals, 13 conference publications (including one poster conference presentation), 10 academic magazine articles, 3 reports, and 2 workshop presentations. (p. 6)

Data Classification/Analysis (p. 6)

The quantitative analysis was used to classify the papers according to the publication year and the type of publication in which the article appeared. Papers were qualitatively classified using open coded content analysis, a technique used by the two studies of literature relating to Twitter (Gao, et al., 2012; Williams, Terras, & Warwick 2013). (p. 6)

Results (p. 7)

The first MOOC related paper was published in 2008, with again just one paper identified in 2009, seven papers in 2010, 10 in 2011, and 26 in 2012. (p. 7)

The majority of identified articles published to date are in journals (17 papers), with a smaller number of articles appearing in conference proceedings and magazines (13 and 10 respectively). (p. 7)

It is noticeable that while many of the publications identified only had one article on MOOCs, exceptionally among the journals the International Review of Research in Open and Distance Learning published six articles on MOOCs, while the European Journal of Open Distance and E-Learning pub lished three. Communications of the ACM published the highest number of magazine articles (three) while eLearn Magazine and Learning Solutions magazine published two articles each. The International Conference on Networked Learning published three articles on MOOCs, two in 2010 and another in 2011. (p. 9)

A list of authors with more than one article being reviewed here are presented in Table 3. (p. 9)

Publication Topics/Themes (p. 11)

The articles embraced a wide-range of themes relating to MOOCs; an initial list included3: • agency • connectivism • actor network theory • dangers • learner experience • pedagogies • technology • trends (p. 11)

The themes were re-stratified into the following categories. 1. Introductory: explaining aspects of MOOCs. 2. Concept: encompassing discussion papers on topics such as the threats and opportunities of MOOCs for Higher Education and its existing institutions. 3. Case studies: examining one or more MOOCs (including papers studying the same course running in different years and papers studying different courses). 4. Educational theory: considering the pedagogic approaches used. 5. Technology: presenting details or consideration of the software and hardware used. 6. Participant focussed: considering aspects related to the learners participating in MOOCs. 7. Provider focussed: considering aspects related to the provider of the MOOC, including the course creators and leaders. 8. Other: this category was introduced to cover the two articles that did not come under any other category. These were Esposito (2012), which discussed ethical (p. 11)

issues in using data generated through a MOOC, and Frank (2012), which presented the author’s views on alternative ways to run MOOCs. (p. 12)

Case Studies (p. 13)

Across the articles classified, 21 had a case study element, and in total 13 different MOOCs were studied. (p. 13)

Researchers’ role. Seven of the case study articles defined the researcher’s role in relation to the study. In two of these instances the researcher was described as a learner in the MOOC, in another two as a participant, and in the remaining three as an observer. (p. 14)

Participants. The majority of the participants in the case study research were people who had participated in the MOOC under consideration. (p. 15)

Other materials such as blog posts and unpublished reports are not included in this review. (p. 15)

Articles published in languages other than English were not considered for this review. (p. 15)

Despite many research studies using publicly available data from a MOOC for research purposes, only a few papers have considered the ethical aspects of such use. (p. 16)

Gao, F., Luo, T., & Zhang, K. (2012). Tweeting for learning: A critical analysis of research on microblogging in education published in 2008-2011. British Journal of Educational Technology, 43(5), 783-801.

Williams, S., Terras, M., & Warwick, C. (in press). What people study when they study Twitter: Classifying Twitter related academic papers. Journal of Documentation, 69(3), 384-410.