Bodong Chen

Crisscross Landscapes

Notes: Rudolph, J. L. (2014). Why Understanding Science Matters: The IES Research Guidelines as a Case in Point



Citekey: @Rudolph2014-fd

Rudolph, J. L. (2014). Why Understanding Science Matters: The IES Research Guidelines as a Case in Point. Educational Researcher , 43(1), 15–18.



“appropriate empirical evaluation” that can identify those things that “are in fact improvements,” that will in turn “contribute to the bigger pic- ture of scientific knowledge” (p. 11). (p. 1)

s far back as the 1950s (as many education researchers well know), federal efforts to reshape education drew on research practices and organiza- tional approaches from the natural sciences, where instrumental success and cumulative progress are the norms. (p. 1)

In reading the text from the IES call for proposals, the effort to apply a particular vision of science is readily apparent. (p. 1)

unproductive history of educational practice that ebbs and flows without clear direction (p. 1)

If we accept the fact that our research methods are dependent on what it is we are seeking to understand, it makes sense, then, to ask ourselves whether the phenomena of teaching and learn- ing are best examined using an experimental approach (p. 2)

More recently, we have individuals like the Nobel Prize– winning physicist–turned–education researcher, Carl Wieman (2007), who has argued that efforts to produce effective teaching at scalable levels can be had only by applying “practices that are essential components of scientific research” (p. 10). Such prac- tices, he notes, “explain why science has progressed at such a remarkable pace in the modern world” (Wieman, 2007, p. 10). (p. 2)

the kind of knowledge we are able to produce is clearly dependent on the phenomena under study. (p. 2)

IES leaders explicitly drew on exper- imental models from the fields of medicine and agriculture, where randomized controlled trials were the standard (see, e.g., U.S. Department of Education, 2003). (p. 2)

The allure of these scientific research models is obvious. In the face of complex and persistent educational problems, they seem to promise objective results, uniform solutions, and stan- dardized interventions less prone to ideological distortion that will actually “work” in our nation’s classrooms (p. 2)

In the field of education, however, researchers are trying to understand things that are far more complex: the way students learn from text, lecture, visual representations, or combinations thereof. They seek to find out, in addition, how that learning is shaped by prior knowledge and experience and by interactions with teachers and student peers. Complicating the picture is the fact that the object of study in education research is a knowing participant who can resist, cooperate, or simply not engage in the instruction being observed (a character- istic not typically found among subatomic particles or even most mammals). (p. 2)

“Science” for the average citi- zen typically entails some activity centered around experimenta- tion whereby a hypothesis or conjecture is demonstrated to be true (or false) with absolute certainty, often revealing in the pro- cess knowledge of how this or that corner of nature “really works” (Lederman, 1992). (p. 2)

Context matters as well in ways that are irrelevant to physical systems. It is one thing to limit study to what happens within a classroom, but classrooms exist in a broader matrix of institu- tions, political systems, and cultures. Whether and what learning takes place are highly contingent on everything from immediate and long-term educational goals to local and national politics, considerations of the global economy, and the allocation of resources (again, to highlight only a handful of factors). (p. 2)

Science, at least this particular version of it, possesses a level of cultural authority that is unmatched in modern society (p. 2)

This particular view of science, however, represents only a narrow slice of the myriad intellectual, social, and cultural prac- tices that fall under the rubric of science more broadly consid- ered. (p. 2)

I would add that all of this changes over time. (p. 2)

Rather, the work of scientists is distributed among a diverse number of smaller research communities, each of which organically fashions its own set of methods, standards of evi- dence, types of representation, forms of argumentation, social and institutional arrangements, and the like depending on both the nature of the phenomena being studied and the questions deemed worthy of exploring. That is to say that the methods of inquiry are highly contextual, contingent, and emergent over time. (p. 2)

i.e., different paradigms (p. 2)

Clearly, teaching and learning, as they happen in classrooms and lecture halls all over the world—and through time—are far different phenomena from those studied in con- trolled laboratory settings or even in field studies of naturally occurring plant or animal populations. Education, as an empiri- cal phenomenon, is just the sort of context-sensitive, dynami- cally responsive complex system that philosopher of science Sandra Mitchell (2009) argues requires research methods that are locally applied, tolerant of uncertainty, and pragmatically (p. 2)

adopted to meet particular social ends at a given point in time— methods that go well beyond randomized controlled trials. (p. 3)

There are without a doubt aspects of learning and educa- tional practice that are amenable to experimental and quasi- experimental study, and with some questions, real progress can indeed be made. (p. 3)

More work clearly needs to be done to better understand how research, policy, and practice might be most productively integrated for the broader goal of social improvement. Helping the public and poli- cymakers (who, after all, in our democratic political system are members of the public) understand just how various scientific research practices work to generate reliable knowledge seems to be a logical first step. (p. 3)

this is a much needed positive vibe going forward. (p. 3)

But we should rightly be concerned when poli- cymakers (supported by a public operating with an incomplete understanding of what science is and how it works) seek to push particular research models and methodological approaches in a misguided attempt to secure knowledge outcomes (reliable, pre- dictable, uniform, etc.) that are unlikely to be obtained given the nature of the activities and enterprise in question. (p. 3)

These are just the kind of things that fall into what the sociologist of science Thomas Gieryn (1999) has termed “boundary work.” (p. 3)

First, if we accept the fact that physical and educational sys- tems operate at fundamentally different levels of complexity, then any attempt to move education toward a hard-science research model in which we try to measure and/or control key variables is bound to fail at the broad-based societal levels that really matter. Put simply, the things we are able to measure in such a system rarely conform to the learning outcomes we most highly value as a society. (p. 3)

The second possibility relates more directly to the power the experimental model has to shape the world outside the labora- tory. (p. 3)

We would need to, in other words, make the naturally occurring system more like the experimental system, a change that would require the simplification of natural learning environments. (p. 3)