Bodong Chen

Crisscross Landscapes

Notes: An extended systematic review of Canadian policy documents on e-Learning



Citekey: @borokhovski2011extended

Borokhovski, E., Bernard, R., Mills, E., Abrami, P. C., Wade, C. A., Tamim, R., … Surkes, M. A. (2011). An extended systematic review of Canadian policy documents on e-Learning: What we’re doing and not doing. Canadian Journal of Learning and Technology/La Revue Canadienne de L’apprentissage et de La Technologie, 37(3), 1–30.



In total, 138 policy documents from Canadian provinces and territories and several federal agencies, dated from 2000 to 2010, were identified, retrieved and analyzed using prescriptive and emergent coding approaches with the purpose of uncovering and describing areas of commonality and inconsistency and, hence, determining where discussions about e-learning are lacking. The review confirmed that Canadian policy makers view technology as offering various potential benefits to learners, but also revealed a troubling lack of specific details, consistency and coordination in facilitating the development of e-learning to fulfill these optimistic expectations. (p. 1)

Studies of technology integration in classrooms and technology-supported distance education have demonstrated the importance and benefits of technology as a way to equip learners for the future. (p. 3)

To determine whether these positive results yielded action by Canadian policy makers, we conducted a systematic review of policy documents concerning e-learning. This project is an extension of two relatively recent publications (Abrami, Bernard, Wade, Schmid, Borokhovski, et al., 2006; McGreal & Anderson, 2007) that describe e-learning in Canada in two distinct ways. One approach is empirical and the other is non-empirical, and they both make useful contributions to the ongoing dialogue among Canadians on how to increase adoption of e-learning practices. (p. 3)

The current work attempts to further our understanding by providing a more comprehensive and systematic review of e-learning policy documents which goes beyond Abrami et al. (2006) and McGreal and Anderson. (p. 3)

Abrami et al. (2006) study used a systematic review methodology to construct an “argument catalogue,” the results of which were touted as constituting “a rough sketch of the evidence, gaps and promising directions in e-learning from 2000 to 2005, with a particular focus on Canada” (p. 1). (p. 4)

McGreal and Anderson paper, “E-Learning in Canada,” is a narrative description addressing essentially the same issues described empirically by Abrami et al. (2006). (p. 4)

The provinciality of Canadian e-learning serves to highlight the inability of Canada to sustain national strategies and focus, such as those implemented in many other countries, due to the fractious nature of federal and provincial relations, particularly in education. (p. 5)

Method (p. 5)

Search Strategy and Document Selection (p. 5)

Education and training department Web sites and the Council of Ministers of Education, Canada (CMEC) Web site were the primary sources for documents representing 10 provinces and 3 territories. In addition, the review targeted six selected federal departments: Canadian Heritage, National Defence, Health Canada, Industry Canada, Human Resources and Skills Development Canada, and Indian and Northern Affairs Canada. (p. 5)

the following categories: policies, vision statements, initiatives, and strategic plans and guidelines. (p. 5)

A total of 107 documents were retained for review. (p. 6)

The second round of data collection was conducted in August 2010 (covering the period from 2008–2010). (p. 6)

31 additional documents were added to the data. (p. 6)

Coding Procedures (p. 6)

Two complementary approaches were used in coding the included policy documents. One approach used a set of predefined, prescriptive codes, while the second employed an emergent coding scheme allowing the categories to form as documents were read. (p. 6)

n both cases, the unit of analysis was an entire document, regardless how often a particular topic emerged within it. (p. 6)

Argument Catalogue approach (Abrami, Bernard, & Wade, 2006) that seemed to be the best fitting tool for the purposes of the current review. (p. 6)

Prescriptive coding (p. 7)

The prescriptive categories (including relevant sub-categories) used in the coding process were determined through a larger-scale review of the literature undertaken by Abrami and colleagues in 2006, which included the following: (p. 7)

(1) Presumed benefits of e-learning (p. 7)

(2) Support for Implementation. (p. 7)

(3) Collaboration. What kind of collaboration has been suggested or exists among the parties involved? (p. 7)

(4) Major areas of e-learning application (p. 8)

(5) Other (financial and organizational) issues (p. 8)

(6) Type of learners and settings (p. 8)

Emergent coding (p. 9)

The emergent coding procedure was conducted in two phases: a review and extraction phase and a coding phase. (p. 9)

Results (p. 9)

Overall Distribution of Documents (p. 9)

Generally speaking, statements citing the benefits of and support for e-learning made up between 40% and 55% of the statements. Issues relating to collaboration (i.e., consultation, sharing) ranged between an additional 13% and 29% of the statements. The total percentage of statements regarding the details of implementation in the areas of “applications,” “other issues” and “learners,” ranged from 18% to 42% of the total number of policy statements. (p. 9)

Largely in agreement with previous reviews (e.g., Abrami et al., 2006), the “benefits of elearning” category was the most prominent theme in the documents, suggesting that elearning has value as a viable educational tool. (p. 11)

the most frequently mentioned subcategories were flexibility/accessibility (82 references), meeting social demands (78 references), interactivity/communication (67 references), and achievement (59 references). The issues of how e-learning can help reduce attrition/improve retention or improve the cost-efficiency ratio were discussed much less frequently (9 & 13 references, respectively). (p. 11)

The “support for implementation” theme was second in frequency of occurrence. Within this theme, the use of online resources, naturally, was cited most often (68 references). The subcategory of professional development was mentioned 63 times, indicating that policy makers acknowledge and value teachers’ expertise and qualifications. The subcategory of logistic/infrastructure received the same number of references (63). These subcategories were followed by course development (50 references). Need for research or actual use of research findings was mentioned in only 32 documents. The two following forms of support for e-learning were the least frequently mentioned: creation of learning object repositories (22 references) and print-based resources (12 references). (p. 12)

Collaboration, another pre-condition for enhancing and supporting e-learning implementation, was the third most commonly cited theme in the policy documents. Within this theme, the majority of the documents emphasized the importance of the coordination of efforts involved in the preparation for and implementation of e-learning. (p. 13)

While this was true within jurisdictions, there was inconsistency in educational policies with regard to elearning across provinces. (p. 13)

More specifically, coordination of efforts across agencies and institutions was the most prominent topic (74 references), followed by sharing resources (54 references) and involvement of other parties (44 references). (p. 13)

There appeared to be strong consensus in the policy documents reviewed that learning with technology has great potential in a variety of areas. Connectivity with remote learners was most often described as the major objective for learning with technology (mentioned in 63 documents), whereas there were fewer references for literacy (43). Furthermore, numeracy and scientific reasoning were only mentioned 22 and 19 times, respectively. Finally, references to second language learning and home schooling were virtually nonexistent—10 and 5 times, respectively. (p. 14)

Within this category, the issue of financial responsibilities clearly dominated (47 references), followed by content and course standards (30 references) and more general financial considerations (25 references). By comparison, issues of ethics and security, certification and copyright, and enrollment eligibility received much less attention (24, 21, and 10 references, respectively). (p. 15)

Type of Learners This category, supposedly reflecting the major group of e-learning recipients and targeted educational settings, was the least represented topic in the reviewed documents (only 70 references, in total). Within this category rural and Aboriginal schools and special needs students were mentioned most often (26, 19, and 14 references, respectively), while (p. 16)

references to gifted students, for example, only appeared 2 times. The virtually nonexistent percentage of references to the average population of learners and urban schools could be attributable to their perception by policy makers as major recipients of e-learning activities by default, though it does not mean that e-learning implementation for this category of users does not require special attention, preparation, and regulation. (p. 17)

Findings Summary and Discussion (p. 17)

As the review revealed, Canada does not have a comprehensive or coherent approach to align e-learning’s vast potential with a clearly articulated and informed understanding of what it could or should accomplish. Instead, e-learning in Canada consists of loosely connected provincial, territorial and federal e-learning networks, education providers (public and private) and targeted initiatives. The consequences of this approach include duplicated efforts, fragmented goals and objectives, and sporadic and short-term initiatives. (p. 17)

Overall, the data suggest that there is a consensus among policy makers in their positive perception of e-learning and its potential to benefit learners. It is clear from this review that Canadian provincial and federal policy makers view technology as offering a variety of benefits to learners. These benefits include flexibility/accessibility, meeting social demands, interactivity/communication, and learner achievement. The extent of attention paid to support of e-learning (i.e., the conditions that are thought to optimize the chances for successful implementation) is also fairly strong, unlike the other coded categories. More specifically, support for implementation received a high level of attention with emphasis on professional training, Web-based resources, and logistic infrastructure. (p. 18)

However, the use of research to guide and support e-learning implementation was noticeably underrepresented. (p. 18)

The problem of the lack of specifics is also evident with respect to application areas (i.e., in what ways and how much e-learning would benefit the development of literacy, numeracy, scientific reasoning, etc.). These issues were seldom addressed. Also, the types of learners who are targeted by and would potentially benefit the most from e-learning were addressed only minimally by the majority of reviewed documents. Finally, it was observed that little attention was paid to regulation and economic issues pertaining to e-learning. (p. 18)

Following what McGreal and Anderson (2007) describe as “significant trends in Canadian e-learning that should be watched” (p. 4), this study found a number of areas requiring greater attention, such as the development of common standards (either technical or content-related) for establishing learning object repositories, blueprints for policies about credit exchange and acceptance, etc. Interestingly, a much earlier review conducted by Rossiter (2002) identified the steps necessary to implement a Canadian action plan on elearning, including encouraging research in the area of e-learning; identifying best practices; and developing comprehensive standards. Despite these recommendations, the majority of policy documents do not contain adequate insight on these issues. (p. 18)

If so, there is a need for higher transparency and better vocalization of educational policies. (p. 18)

Ideally, policies are intended to translate insights and consensuses based on research findings, public opinion and the balanced evaluation of needs and resources into realistic action plans. This review is just the first step in attempting to understand the current state of e-learning policy in Canada and how much of it has been lost in translation. (p. 19)

Abrami, P. C., Bernard, R. M., Wade, C. A., Schmid, R .F., Borokhovski, E., Tamim, R., Surkes, M., Lowerison, G., Zhang, D., Nicolaidou, I., Newman, S., Wozney, L., & Peretiatkowicz, A. (2006). A review of e-learning in Canada: A rough sketch of the evidence, gaps and promising directions. Canadian Journal of Learning and Technology. 32(3), 1-78. (p. 19)