Bodong Chen

Crisscross Landscapes

Notes: Wieman, C. E. (2014). The Similarities Between Research in Education and Research in the Hard Sciences



Citekey: @Wieman2014-vd

Wieman, C. E. (2014). The Similarities Between Research in Education and Research in the Hard Sciences. Educational Researcher , 43(1), 12–14.



Here I would like to offer a different perspective, how there is a considerable degree of similarity between research in the hard sciences and education, and how this provides a useful lens for thinking about what constitutes “rigorous” and “scientific” education research. (p. 1)

A fundamental test of research in the hard sciences is, does the result have predictive power? By that, I mean can one use the results to predict with reasonable accuracy what will happen, or what will be observed, in some new situation (at a minimum in a replication of the experiment as described by the original researcher)? This standard has served the hard sciences well over the years and, I argue, is correspondingly useful to use for educa- tion research. (p. 1)

the nature of research in the hard sciences is often misunderstood and mischaracterized. True research in the hard sciences, when it is exploring fundamentally new ground, is much messier, complicated, and less precise than is usually recognized and, thus, more similar to education research. The errors that lead to flawed research also have much the same origin across the differ- ent fields. (p. 1)

There is the messiness of having many, many quantities that “might” be important, and the experimental results obtained in such circumstances (p. 2)

related to ‘correlational’ research (p. 2)

Physics and chemistry are quite mature sciences, and so most of the research that gets presented in textbooks, classes, and even in the media has all this messiness understood and cleaned up. It is much like seeing a child only through official portraits taken after they are grown, cleaned up, and dressed in formal wear, rather than seeing them as they really were, climbing trees and splashing through mud puddles. (p. 2)

It means only that one should be able to pre- dict some meaningful measureable outcomes. This is also not a criterion for the importance of the research. Importance depends on a number of other criteria that vary with field and personal opinions. (p. 2)

For example, a good qualitative study that exam- ines only a few students or teachers in depth will allow one to recognize, and hence more accurately predict, some factors that will be important in educational outcomes and important in the design of larger quantitative experiments in similar populations. Such qualitative research provides an important contribution to the knowledge base, albeit of a different sort than a randomized controlled trial that tests the impact of a large-scale intervention on multiple school districts. (p. 2)

Only a small fraction of what I did in my 30 years of physics research falls in this cutting-edge “messy and complex” cate- gory, and my fraction is probably larger than that for most physicists or chemists. In contrast, much of modern biology is working in much less well-studied and less understood areas of research, and there, the results tend to be far less incremental and correspondingly less reproducible, because the complexi- ties of the systems involved have not yet been so well studied and understood. In general, education research is more like biology research. (p. 2)

Fun and easier in the sense there is so much unplowed ground, so many unanswered questions, and so many potential experiments and possible surprises. Of course, in other respects, it is harder; for example, we know a lot more about the contextual influences on the behavior of atoms than on students and, hence, what contextual elements do and do not have to be controlled in designing experiments. Also, atoms do not require institutional review board approval and consent forms. (p. 2)

The way research goes bad is also quite similar between the hard sciences and education. (p. 2)

The serious errors in hard science research occur when important variables are overlooked, and this is also true in education research. Usually these variables are overlooked for the same reasons in all fields; the researcher is just sloppy, or more often, the researcher is failing to adequately address his or her inherent biases. (p. 2)

n all types of research, it is essential to recognize these inherent biases and to have tests and procedures to prevent those biases from unduly influencing the results and conclusions. (p. 2)

There is an enormous number of ways to get the wrong answer by overlooking some relevant variable, and the mark of a good researcher is to recognize, with limited information, which variables are relevant. Figuring out what to measure, and how well to measure it, is critical in all fields. (p. 2)

Although the common perception is that experimental research in something like physics is much more controlled and precise than in education, and hence such errors are inherently easier to avoid, I do not believe that is actually the case. (p. 2)

When an area of physics research is truly cutting-edge, pushing advances in very new directions where the behaviors and likely outcomes are quite unknown, that is a very different situation. Then, just like in education, the researchers are struggling to figure out what factors are important and how to control or adequately measure what they hope are the relevant (p. 2)

On the other hand, it is possible to be too careful. If a researcher is determined to examine, measure, and carefully con- trol every conceivable variable, he or she will be a failure, be it in hard sciences or education, because he or she will never finish (p. 2)

The measure of a great researcher is the one who understands how to do enough, and only just enough, to obtain important results that are reproducible and have adequate pre- dictive power to advance the field. (p. 3)

In both cases, a basic standard for research should be the predic- tive power of the results; and in both cases, the underlying basic intellectual challenges in experimental design, and reasons for flawed research, are much the same. (p. 3)