Bodong Chen

Crisscross Landscapes

Notes: Johnson2004-cv



Citekey: @Johnson2004-cv







The purposes of this article are to position mixed methods research (mixed research is a synonym) as the natural complement to tradi- tional qualitative and quantitative research, to present pragmatism as offering an attractive philosophical partner for mixed methods re- search, and to provide a framework for designing and conducting mixed methods research. (p. 1)

Qualitative purists (also called constructivists and interpretivists) reject what they call positivism. They argue for the superiority of constructivism, idealism, relativism, humanism, hermeneutics, and, sometimes, postmodernism (Guba & Lincoln, 1989; Lincoln & Guba, 2000; Schwandt, 2000; Smith, 1983, 1984). These purists contend that multiple-constructed realities abound, that time- and context-free generalizations are neither desirable nor possible, that research is value-bound, that it is impossible to dif- ferentiate fully causes and effects, that logic flows from specific to general (e.g., explanations are generated inductively from the data), and that knower and known cannot be separated because the subjective knower is the only source of reality (Guba, 1990). (p. 1)

Both sets of purists view their paradigms as the ideal for re- search, and, implicitly if not explicitly, they advocate the in- compatibility thesis (Howe, 1988), which posits that qualitative and quantitative research paradigms, including their associated methods, cannot and should not be mixed. (p. 1)

Quantitative purists (Ayer, 1959; Maxwell & Delaney, 2004; Popper, 1959; Schrag, 1992) articulate assumptions that are con- sistent with what is commonly called a positivist philosophy.3, 4 (p. 1)

A disturbing fea- ture of the paradigm wars has been the relentless focus on the differences between the two orientations. Indeed, the two dom- inant research paradigms have resulted in two research cultures, “one professing the superiority of ‘deep, rich observational data’ and the other the virtues of ‘hard, generalizable’ … data” (Sieber, 1973, p. 1335). (p. 1)

According to this school of thought, educational re- searchers should eliminate their biases, remain emotionally de- tached and uninvolved with the objects of study, and test or empirically justify their stated hypotheses. These researchers have traditionally called for rhetorical neutrality (p. 1)

We hope the field will move beyond quantitative versus qualitative research arguments because, as recognized by mixed methods research, both quantitative and qualitative research are important and useful. (p. 1)

We contend that epistemological and methodological pluralism should be promoted in educational research so that researchers are informed about epistemological and methodological possibilities and, ultimately, so that we are able to conduct more effective re- search. (p. 2)

increasingly inter- disciplinary, complex, and dynamic; (p. 2)

Meth- odological work on the mixed methods research paradigm can be seen in several recent books (Brewer & Hunter, 1989; Creswell, 2003; Greene, Caracelli, & Graham, 1989; Johnson & Christensen, 2004; Newman & Benz, 1998; Reichardt & Rallis, 1994; Tashakkori & Teddlie, 1998, 2003). Much work remains to be undertaken in the area of mixed methods research regarding its philosophical positions, designs, data analysis, va- lidity strategies, mixing and integration procedures, and ratio- nales, among other things. (p. 2)

Although many research procedures or methods typically have been linked to certain paradigms, this linkage between research paradigm and research methods is nei- ther sacrosanct nor necessary (Howe, 1988, 1992). For example, qualitative researchers should be free to use quantitative meth- ods, and quantitative researchers should be free to use qualitative methods. (p. 2)

the argument is sensible but not convincing enough. (p. 2)

Commonalities Among the Traditional Paradigms (p. 2)

Philosophical Issues Debates (p. 2)

As noted by Onwuegbuzie and Teddlie (2003), some individuals who engage in the qualitative versus quantitative paradigm debate appear to confuse the logic of justification with research methods. That is, there is a tendency among some researchers to treat epistemology and method as being synonymous (Bryman, 1984; Howe, 1992). (p. 2)

Sechrest and Sidani (1995, p. 78) point out that both methodologies “describe their data, construct explana- tory arguments from their data, and speculate about why the outcomes they observed happened as they did.” (p. 2)

this is when terms get mixed up – paradigms and methodologies are both used to describe qual and quan. (p. 2)

For example, differences in epistemologi- cal beliefs (such as a difference in beliefs about the appropriate logic of justification) should not prevent a qualitative researcher from utilizing data collection methods more typically associated with quantitative research, and vice versa. (p. 2)

paradigmatic orientation (p. 2)

n the social and behavioral sciences, this goal of under- standing leads to the examination of many different phenomena, including holistic phenomena such as intentions, experiences, at- titudes, and culture, as well as more reductive phenomena such as macromolecules, nerve cells, micro-level homunculi, and bio- chemical computational systems (de Jong, 2003). (p. 2)

For example, on the “positivist” side of the fence, the barriers that quantitative educational researchers have built arise from a narrow definition of the concept of “science.” 6 (p. 2)

Basic agreement has been reached on each of the following issues: (a) the relativity of the “light of reason” (i.e., what appears reasonable can vary across per- sons); (b) theory-laden perception or the theory-ladenness of facts (i.e., what we notice and observe is affected by our background knowledge, theories, and experiences; in short, observation is not a perfect and direct window into “reality”); © underdeter- mination of theory by evidence (i.e., it is possible for more than one theory to fit a single set of empirical data); (d) the Duhem- Quine thesis or idea of auxiliary assumptions (i.e., a hypothesis cannot be fully tested in isolation because to make the test we also must make various assumptions; the hypothesis is embedded in a holistic network of beliefs; and alternative explanations will continue to exist); (e) the problem of induction (i.e., the recogni- tion that we only obtain probabilistic evidence, not final proof in empirical research; in short, we agree that the future may not re- semble the past); (f) the social nature of the research enterprise (i.e., researchers are embedded in communities and they clearly have and are affected by their attitudes, values, and beliefs); and (g) the value-ladenness of inquiry (this is similar to the last point but specifically points out that human beings can never be com- pletely value free, and that values affect what we choose to in- vestigate, what we see, and how we interpret what we see). (p. 3)

Qualitative researchers also are not immune from constructive criticism. Some qualitative purists (e.g., Guba, 1990) openly admit that they adopt an unqualified or strong relativism, which is logically self-refuting and (in its strong form) hinders the de- velopment and use of systematic standards for judging research quality (p. 3)

Pragmatism as the Philosophical Partner for Mixed Methods Research (p. 3)

Generally speaking, subjective states (i.e., cre- ated and experienced realities) that vary from person to person and that are sometimes called “realities” should probably be called (for the purposes of clarity and greater precision) multiple perspectives or opinions or beliefs (depending on the specific phe- nomenon being described) rather than multiple realities (Phillips & Burbules, 2000). (p. 3)

Mixed methods research should, instead (at this time), use a method and philosophy that attempt to fit together the in- sights provided by qualitative and quantitative research into a workable solution. Along these lines, we advocate consideration of the pragmatic method of the classical pragmatists (e.g., Charles Sanders Peirce, William James, and John Dewey) as a way for researchers to think about the traditional dualisms that have been debated by the purists. (p. 3)

We agree with qualitative researchers that value stances are often needed in research; however, it also is important that research is more than simply one researcher’s highly idiosyncratic opinions written into a report. (p. 3)

many strategies are recognized and regularly used in qualitative research (such as member check- ing, triangulation, negative case sampling, pattern matching, ex- ternal audits) to help overcome this potential problem and produce high-quality and rigorous qualitative research. (p. 3)

providing an adequate rationale for interpretations of their data (Onwuegbuzie, 2000), and qualitative methods of analyses too “often remain private and unavailable for public inspection” (Constas, 1992, p. 254). (p. 3)

We reject an incompatibilist, either/or approach to paradigm selection and we recommend a more pluralistic or compatibilist approach. (p. 4)

Building on Peirce’s lead, James (1995, 1907 original) argued that “The pragmatic method is primarily a method of settling metaphysical disputes that otherwise might be interminable… . The pragmatic method in such cases is to try to interpret each notion by tracing its respective practical con- sequences” (p. 18). (p. 4)

Dewey (1948, 1920 original) stated that “in order to discover the meaning of the idea [we must] ask for its consequences” (p. 132). In short, when judging ideas we should consider their empirical and practical consequences. (p. 4)

in Table 1, what we believe are classical pragmatism’s most general and important characteristics. (p. 4)

Although we endorse pragmatism as a philosophy that can help to build bridges between conflicting philosophies, pragma- tism, like all current philosophies, has some shortcomings. In Table 2 we present some of these. (p. 4)

Practic- ing researchers (p. 4)

Comparing Qualitative, Quantitative, and Mixed Methods Research (p. 4)

Mixed methods research is formally defined here as the class of re- search where the researcher mixes or combines quantitative and qual- itative research techniques, methods, approaches, concepts or language into a single study. (p. 4)

We are advocating a needs-based or contingency approach to research method and concept selection. (p. 4)

Its logic of inquiry includes the use of induction (or discovery of patterns), deduction (testing of theories and hypotheses), and abduction (uncovering and relying on the best of a set of explanations for understanding one’s results) (e.g., de Waal, 2001). (p. 4)

it offers an immediate and useful middle position philosophically and meth- odologically; it offers a practical and outcome-oriented method of inquiry that is based on action and leads, iteratively, to further action and the elimination of doubt (p. 4)

What is most fundamental is the research question— research methods should follow research questions (p. 4)

Prefers action to philosophizing (pragmatism is, in a sense, an anti-philosophy). (p. 5)

Gaining an understanding of the strengths and weaknesses of quantitative and qualitative research puts a researcher in a posi- tion to mix or combine strategies and to use what Johnson and Turner (2003) call the fundamental principle of mixed research. (p. 5)

corroborated (p. 6)

many other typologies (especially Creswell, 1994; Morgan, 1998; Morse, 1991; Patton, 1990; and Tashakkori & Teddlie, 1998) (p. 6)

one can construct mixed-model designs by mixing qualitative and quantitative approaches within and across the stages of research (p. 6)

Tables 3 and 4 are specifically designed to aid in the con- struction of a combination of qualitative and quantitative re- search. After determining one’s research question(s), one can decide whether mixed research offers the best potential for an an- swer; if this is the case, then one can use the tables as an aid to help in deciding on the combination of complementary strengths and nonoverlapping weaknesses that is appropriate for a partic- ular study. Table 5 shows some of the strengths and weaknesses of mixed methods research, which should aid in the decision to use or not use a mixed methods research approach for a given re- search study. (p. 6)

According to Morgan (1998) and Morse (1991), one also may consider the dimension of paradigm emphasis (de- ciding whether to give the quantitative and qualitative compo- nents of a mixed study equal status or to give one paradigm the dominant status). (p. 6)

Development of a Mixed Methods Research Typology (p. 6)

Our mixed-method designs (discussed below) are based on the crossing of paradigm emphasis and time ordering of the quantitative and qualitative phases. (p. 6)

To construct a mixed-method design, the researcher must make two primary decisions: (a) whether one wants to operate largely within one dominant paradigm or not, and (b) whether one wants to conduct the phases concurrently or sequentially. (p. 7)

the degree of mixture, which would form a continuum from monomethod to fully mixed methods. Another dimen- sion pertains to where mixing should occur (e.g., in the objec- tive[s], methods of data collection, research methods, during data analysis, data interpretation). Yet another important dimension is whether one wants to take a critical theory/ transformative-emancipatory (Mertens, 2003) approach or a less explicitly ideological approach to a study. (p. 7)

mixed-method designs are similar to conducting a quantitative mini-study and a qualitative mini-study in one overall research study (p. 7)

Toward a Parsimonious Typology of Mixed Research Methods (p. 7)

two major types of mixed methods research: mixed-model (mixing qualitative and quantitative approaches within or across the stages of the research process) and mixed- method (the inclusion of a quantitative phase and a qualitative phase in an overall research study). (p. 7)

The point is for the researcher to be creative and not be limited by the designs listed in this article. (p. 7)

A tenet of mixed methods research is that researchers should mindfully create designs that effectively answer their re- search questions; this stands in contrast to the common approach in traditional quantitative research where students are given a menu of designs from which to select. (p. 7)

A Mixed Methods Research Process Model (p. 8)

Our mixed methods research process model comprises eight dis- tinct steps: (1) determine the research question; (2) determine whether a mixed design is appropriate; (3) select the mixed- method or mixed-model research design; (4) collect the data; (5) analyze the data; (6) interpret the data; (7) legitimate the data; and (8) draw conclusions (if warranted) and write the final report. (p. 8)

Three steps in the mixed methods research process warrant some further discussion, especially purpose (Step 2), data analy- sis (Step 5), and legitimation (Step 7). (p. 8)

Data integration characterizes the final stage, whereby both quantitative and qualitative data are integrated into either a coherent whole or two separate sets (i.e., qualitative and quantitative) of coherent wholes. (p. 9)

(a) triangulation (i.e., seeking conver- gence and corroboration of results from different methods and designs studying the same phenomenon); (b) complementarity (i.e., seeking elaboration, enhancement, illustration, and clarifi- cation of the results from one method with results from the other method); © initiation (i.e., discovering paradoxes and contra- dictions that lead to a re-framing of the research question); (d) de- velopment (i.e., using the findings from one method to help inform the other method); and (e) expansion (i.e., seeking to ex- pand the breadth and range of research by using different meth- ods for different inquiry components). (p. 9)

The legitimation step involves assessing the trustworthiness of both the qualitative and quantitative data and subsequent inter- pretations. (p. 9)

Quantitative Legitimation Model (Onwuegbuzie, 2003 (p. 9)

Qualitative Legitimation Model (Onwuegbuzie, 2000; Onwuegbuzie, Jiao, & Bostick, 2004 (p. 9)

According to these authors, the seven data analysis stages are as follows: (a) data reduction, (b) data display, © data transformation, (d) data cor- relation, (e) data consolidation, (f) data comparison, and (g) data integration. (p. 9)

Data reduction involves reducing the dimensionality of the qualitative data (e.g., via exploratory thematic analysis, memoing) and quantitative data (e.g., via descriptive statistics, exploratory factor analysis, cluster analysis). Data display, in- volves describing pictorially the qualitative data (e.g., matrices, charts, graphs, networks, lists, rubrics, and Venn diagrams) and quantitative data (e.g., tables, graphs). (p. 9)

The Future of Mixed Methods Research in Education (p. 9)

data transformation stage, wherein quantitative data are converted into narrative data that can be analyzed qualitatively (i.e., qualitized; Tashakkori & Teddlie, 1998) and/or qualitative data are converted into numerical codes that can be represented statistically (i.e., quantitized; Tashakkori & Teddlie, 1998). Data correlation involves the quantitative data being correlated with the qualitized data or the qualitative data being correlated with the quantitized data. (p. 9)

data consolidation, wherein both quantitative and qualitative data are combined to create new or consolidated variables or data sets. (p. 9)

In general we recommend contingency theory for research approach selection, which accepts that quantitative, qualitative, and mixed research are all superior under different cir- (p. 9)

FIGURE 3. Mixed research process model. (p. 10)

cumstances and it is the researcher’s task to examine the specific contingencies and make the decision (p. 10)

By narrowing the divide between quanti- tative and qualitative researchers, mixed methods research has a great potential to promote a shared responsibility in the quest for attaining accountability for educational quality. (p. 11)