read

References

Citekey: @Veletsianos2015-hd

Veletsianos, G., & Shepherdson, P. (2015). Who studies MOOCs? Interdisciplinarity in MOOC research and its changes over time. The International Review of Research in. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/2202

Notes

Summarize: This study builds on earlier studies of the MOOC literature, but with a special focus (on interdisciplnarity) and a different approach. Briefly, it studies the scope of the literature from 2013-2015; it uses author affiliation as a proxy for Interdisciplinarity; it covers a broad range of literature extracted from multiple indices; it relies on descriptive and inferential statistics (Chi-square) to describe interdisciplinarityand compare it with data from earlier studies.

Assess: A well-planned study and a well-crafted paper overall. Also, sincere explanation of the limitations with the study. The treatment of interdisciplnarity in this context is convincing even though I had doubts when diving into this article.

Reflect: This paper builds upon earlier ones and points to interesting future directions to be explored. Compared to Gasevic et al. (2014), which uses a different dataset and very different analytic approaches, this study’s techniques are relatively simple, but to the point. Future directions could follow deepening the analysis conducted here, by, for instance, using citation data (bibio coupling); or by mashing-up with other data sources, such as Twitter data at conferences.

Highlights

We apply descriptive and inferential statistics to bibliometric data to investigate interdisciplinarity in MOOC research. Results show that MOOC research published in 2013-2015 was (a) mostly conducted by researchers affiliated with Education and Computer Science disciplines, (b) far from monolithic, (c) had a greater representation of authors from Computer Science than in the past, and (d) showed a trend toward being more interdisciplinary than MOOC research published in 2008-2012. Our results also suggest that empirical research on xMOOCs may be more interdisciplinary than research on cMOOCs. Greater interdisciplinarity in xMOOC research could reflect the burgeoning interest in the field, the general familiarity with the xMOOC pedagogical model, and the hype experienced by xMOOCs. (p. 1)

Makes great sense. Also start wondering whether we have to promote interdisciplinarity at all, regardless its “benefits.” (p. 1)

Nevertheless, in their assessment of proposals submitted for funding under the MOOC research initiative (henceforth MRI), Gašević, Kovanović, Joksimović, and Siemens (2014) show that more than 50% of the authors in all phases of the MRI grants were from the field of education, even though a common perception in the field is that the MOOC phenomenon is “driven by computer scientists” (p. 166). Research into emerging forms of digital learning is likely to suffer if driven by education researchers alone or computer scientists alone. (p. 2)

We have recently completed a systematic review of the empirical MOOC literature published between 2013 and 2015 (Veletsianos & Shepherdson, 2015), and as a result of that study, we can use bibliometric data to investigate the extent to which interdisciplinarity is present in the published literature on MOOCs. Thus, in this paper, we combine our data with data from Gašević et al. (2014), and data used in a past systematic review of the literature ( Liyanagunawardena, Adams, & Williams, 2013), to examine interdisciplinarity in the MOOC literature, and whether and how this has changed over time. (p. 2)

Literature Review (p. 3)

Interdisciplinary research is “any study or group of studies undertaken by scholars from two or more distinct scientific disciplines” (Aboelela et al., 2007, p. 341). The value of interdisciplinary research and collaboration rests in both the complexity of the problems societies are faced with (Pellmar & Eisenberg, 2000), and in the novel perspectives that academic “immigrants” can bring to a field other than their own (Nissani, 1997). (p. 3)

Research investigating interdisciplinarity has explored its utility in academic, educational, and practical contexts (Choi & Pak, 2006), factors mitigating against its occurrence (Bauer, 1990), and whether indications of its presence actually reflect cross-pollination between different disciplines (Schummer, 2004). (p. 3)

Huang and Chang (2011) demonstrated that research in Information Science has become increasingly interdisciplinary over the course of three decades. (p. 3)

he scientific complexity facing research into digital and online education is increasingly expanding. This is evident in the context of research into MOOCs and networked learning/participation. (p. 3)

Zuckerman, Azari, and Doane, (2013, p. 17) for instance argue that the learning sciences and learning technology communities are at the cusp of catalyzing potentially transformative change in post-secondary education…At the same time, there are many unknowns regarding the fundamental science of learning, the translation of those fundamental ideas into curriculum design (p. 3)

Eisenhart and DeHaan (2005) proposed that doctoral preparation programs for education researchers include interdisciplinary collaborations as one of their core components, and the scientific community has responded to the scientific complexity of digital education in part by developing the field of learning analytics, which Gašević, Dawson, and Siemens (2015, p. 64) describe as a “bricolage field drawing on research, methods, and techniques from numerous disciplines such as learning sciences, data mining, information visualization, and psychology.” (p. 4)

While we have been able to identify some papers encouraging researchers to cross disciplinary lines to investigate digital education (e.g., Alavi & Leidner, 2001), studies exploring the degree to which digital education research in general, and MOOC research in particular, is interdisciplinary, are scant. (p. 4)

The one study that we have i dentified that examined multidisciplinarity in the MOOC context was the analysis of funding submissions to the MOOC research initiative (MRI) by Gašević et al. (2014). (p. 4)

However, given that their analysis focused solely on submissions seeking funding from a specific source, the extent to which this is reflective of MOOC research more broadly is unclear. (p. 4)

While not directly bearing upon the topic of interdisciplinarity in MOOC research, a study by Kirby, Hoadley, and Carr-Chellman (2005) is also informative. These authors were interested in understanding the relationships between the Instructional Systems Design and Learning Sciences fields - two fields of study interested in the use of technology for learning - and conducted a citation analysis to identify these relationships. (p. 4)

a bibliometric app roach does allow us to describe the interdisciplinary nature of MOOC research as regards Aboelela et al.’s (2007) definition of the term: studies undertaken by scholars from different disciplines. Specifically, using the papers collected for a systematic review of the literature reported in Veletsianos and Shepherdson (2015) provides an opportunity to assess the disciplinary distribution of authors conducting recent research into MOOCs. (p. 4)

Research Questions (p. 5)

RQ 1: What are the disciplinary backgrounds of the authors who published empirical MOOC research in 2013-2015? RQ 2: How does the disciplinary distribution of the authors who published MOOC research in 2013-2015 compare to that of the submissions to the MRI reported by Gašević et al. (2014)? RQ 3: Is the 2013-2015 empirical research on MOOCs more or less interdisciplinary than was previously the case? (p. 5)

Methods (p. 5)

an exhaustive description of the methodology used to systematically gather empirical literature on MOOCs is presented by Veletsianos and Shepherdson (2015) (p. 5)

To discover empirical MOOC literature we conducted searches using the key words “MOOC” or “Massive Open Online Course” and limited our results to 2013-2015. These searches occurred between January 7, 2015 and February 1, 2015. Our inclusion criteria were the following: To be included, the identified document ought to focus on MOOCs and ought to have been ( 1) empirical, (2) published in a peer-reviewed journal, in conference proceedings, or in Educause Review, (3) published or was available online as in press between January 2013 and January 2015, and (4) written in English. Three researchers were involved in this process and they read each abstract to identify papers for inclusion. If no decision could be made by examining the abstract, the complete paper was downloaded and examined. The databases, search engines and journals used in this search were the following: Scopus, Journal of Online Learning and Teaching, Summon, Google Scholar, EdITLib Digital Library, and Educause Library. The Journal of Online (p. 5)

Learning and Teaching was searched because it was not indexed by Scopus, and because it was also examined by Liyanagunawardena et al. (2013) in their own systematic review of the MOOC literature. 120 empirical papers that fit the inclusion criteria were identified via these search strategies. (p. 6)

At this point, a forward referencing search strategy and a reference list search strategy were also used to discover more literature. (p. 6)

Data Collection (p. 6)

•Names and affiliations of each author (n=462) who had contributed to at least one paper in the corpus consisting of empirical MOOC research published in 2013-2015; •Affiliation data reported by Gašević et al., (2014) (n=429); and •Names and affiliations of each author (n=56) who had contributed to at least one paper in the corpus identified by Liyanagunawardena et al. (2013). (p. 6)

Data Analysis (p. 6)

RQ 1: What are the disciplinary backgrounds of the authors who published empirical MOOC research in 2013-2015? After collation of author affiliation information was complete, two independent raters allocated each affiliation to one of a list of 12 disciplinary categories we devised during the collation process. (p. 7)

RQ 2: How does the disciplinary distribution of the authors who published MOOC research in 2013-2015 compare to that of the submissions to the MRI reported by Gašević et al. (2014)? We used Gašević et al.’s reported disciplinary frequencies for the top five disciplines to which MRI submitters belonged, and compared these to the frequencies of the same disciplines in our corpus by way of a chi-square test. (p. 7)

RQ 3: Is the 2013-2015 empirical research on MOOCs more or less interdisciplinary than was previously the case? To determine changes in MOOC research interdisciplinarity, we first undertook the same affiliation categorisation process with the corpus of literature collected in a systematic review of MOOC literature from 2008–2012 by Liyanagunawardena et al. (2013) as we had undertaken with our corpus. We then used two statistical measures to assess differences between the corpora. (p. 7)

First, we used a chi-square test to compare the relative frequency of each discipline in the two corpora. Second, we used Brillouin’s index (Brillouin, 1956) to assess the overall interdisciplinarity of each corpus. This index is used as a measure of population diversity in the field of ecology (e.g., Peet, 1975), and has been used to assess interdisciplinarity in other research fields (e.g., Huang & Chang, 2011; Steele & Stier, 2000). (p. 7)

Findings (p. 8)

RQ 1: What are the disciplinary backgrounds of the authors who published empirical MOOC research in 2013-2015? (p. 8)

Frequencies for each discipline in our corpus are displayed in Table 1. Education and Computer Science were the two most frequent affiliations, and between them accounted for over half of the authors. Industry, Social Sciences, Engineering, Medicine, and Administration each occurred in greater than 5% of the affiliations, with all remaining disciplines occurring with relatively less frequency. (p. 8)

the field appears to be far from monolithic. (p. 8)

RQ 2: How does the disciplinary distribution of the authors who published MOOC research in 2013-2015 compare to that of the submissions to the MRI reported by Gašević et al. (2014)? (p. 10)

Chi-square tests showed that our corpus and that of Gašević et al., (2014) differed significantly [X2(4,n = 781) = 39.57, p < .001 ], with standardised residuals indicating that our corpus had a greater representation of authors from Computer Science (standardised residual of 5.69), and the Gašević et al. corpus a greater representation of authors from Education and Industry (standardised residuals of 4.31 and 1.95, respectively). (p. 10)

RQ 3: Is the 2013-2015 empirical research on MOOCs more or less interdisciplinary than was previously the case? (p. 10)

Brillouin’s index indicated increased interdisciplinarity in our corpus (H B = 1.99 ) compared to that of Liyanagunawardena et al. (2013) (H B = 1.52). Results of the bootstrapping process showed a trend toward greater interdisciplinarity in our corpus (p = .095). (p. 10)

Discussion (p. 10)

Second, future research could examine interdisciplinarity using alternative approaches. For instance, researchers could examine (a) the degree of interaction and awareness between the two disciplines by using methods similar to the ones used by Kirby, Hoadley, and Carr-Chellman (2005), or (b) the extent to which published research features interaction between the two fields not just in cross-citations of MOOC literature, but also in the ways that separate bodies of knowledge are integrated and combined in the design, analysis, and reporting of the research. Finally, future research could also examine whether fragmentation in the community is changing over time (e.g., by conducting the analysis described in a and b above for L@S 2014 and L@S 2015 and comparing the results). (p. 11)

A positive trend uncovered by our investigation is that research on MOOCs appears to be more interdisciplinary than in the past, suggesting that the scientific complexity of the field is being tackled by a greater diversity of researchers. This suggests that even though xMOOCs are often disparaged for their teacher-centric and cognitivist-behaviorist approach (Hew & Cheung, 2014; Kennedy, 2014), empirical research on xMOOCs may be more interdisciplinary than research on cMOOCs, which are deemed to embrace more progressive learning designs (Rodriguez, 2012). We make this statement on the basis of the fact that two distinct “phases” of the MOOC phenomenon appear to exist (cMOOC and xMOOC, as identified by Ebben and Murphy, 2014) and: (a) the Liyanagunawardena et al. (2013) corpus focused on cMOOCs, (b) the overwhelming majority of the studies included in our corpus focused on xMOOCs, and (c) our findings show that our corpus (p. 11)

was more interdisciplinary than that of Liyanagunawardena et al.2 Greater interdisciplinarity in the field represents a positive trend for digital education research. (p. 12)

Finally, although our corpus had a greater representation of authors from Computer Science than was apparent in earlier MOOC research, our results may have been influenced by disciplinary norms in publishing. (p. 12)

As such, the disciplinary makeup of our corpus may reflect differences in publication cycles between the publication outlets preferred by different disciplines. Unfortunately, the data we have gathered do not allow us to examine this insight empirically; nonetheless, this may be a worthwhile topic for future research. (p. 12)

Limitations (p. 13)

Even though we used authors’ disciplinary affiliations as a proxy of their home discipline and we believe this to be a strong proxy, one limitation facing this study is the fact that authors may draw from a variety of disciplines in their research. (p. 13)

A second limitation that readers need to be aware of reflects variations between the composition of our corpus (empirical papers) vis-a-vis Liyanagunawardena et al.’s (2013) corpus (papers that were both empirical and otherwise). (p. 13)

Education and digital learning researchers may need to (a) take on a more active role in educating colleagues from other disciplines about what education researchers do and do not know about digital learning from the research that exists in the field and, (b) remain open to the perspectives that academic “immigrants” can bring to this field (cf. Nissani, 1997). At the same time, researchers from fields other than education need to recognize that education research and educational technology research have a long and rich history and an empirical basis that can inform design and research decisions. Working together, multidisciplinary, interdisciplinary, and crossdisciplinary teams of researchers can improve our scholarly understanding of teaching and learning. (p. 14)

Revisiting my question in the beginning of this article, I think authors have made a strong point for interdisciplinarity in MOOC research. Well done. (p. 14)

Blog Logo

Bodong Chen


Published

Image

Crisscross Landscapes

Bodong Chen, University of Minnesota

Back to Home