Bodong Chen

Crisscross Landscapes

Notes: Tong - 2014 - A Randomized Study of a Literacy-Integrated Science Intervention



Citekey: @Tong2014-tx

Tong, F., Irby, B. J., Lara-Alecio, R., Guerrero, C., Fan, Y., & Huerta, M. (2014). A Randomized Study of a Literacy-Integrated Science Intervention for Low-Socio-economic Status Middle School Students: Findings from first-year implementation. International Journal of Science Education, 36(12), 2083–2109.



This paper presents the findings from a randomized control trial study of reading/literacy-integrated science inquiry intervention after 1 year of implementation and the treatment effect on 5th-grade low-socio-economic African-American and Hispanic students’ achievement in science and English reading. (p. 2)

A total of 94 treatment students and 194 comparison students from four randomized intermediate schools participate (p. 2)

how were schools randomized? (p. 2)

In this study, we examined the effect of a one-year literacy-integrated science intervention delivered to fifth-grade Hispanic and African-American low- SES male and female students on their science and reading achievement. (p. 4)

Intent (p. 4)

Science, Literacy and Ethnicity, SES, and Gender (p. 4)

The 2011 National Assessment of Educational Progress (NAEP) (NCES, 2012b) data show that such ethnically diverse students score lower than mainstream students on science assessments—tests requiring scientific literacy. (p. 4)

SES was also a significant factor associated with reading achievement. (p. 4)

According to Lee (2005), there is a noted a lack of cultural congruence between the students’ home culture and language and the westernized ideals and language of science in the literature related to ethnically diverse students, who are more likely to come from low-SES families. (p. 4)

The situation of gender disparity in science achievement is actually more serious for diverse, low-SES students. (p. 5)

Other researchers also have provided converging evidence that the gender gap in science achievement starts as early as elementary grade (Mullis, Dossey, Owen, & Phillips, 1993) and keeps increasing at the pre-high-school level (Drew, 1996). (p. 5)

Science Inquiry (p. 6)

The new conceptual framework for the Next Generation Science Standards (NGSS, 2012) promotes the use of science inquiry in science teaching and learning as scien- tific thinking and discourse are developed. (p. 6)

Science inquiry has been successfully applied in increasing scientific understanding among ethnically diverse, low-SES students (Cuevas, Lee, Hart, & Deaktor, 2005)as well as promoting academic achievement over time (Amaral, Garrison, & Klentschy, 2002). (p. 7)

Therefore, it is critical to integrate literacy instruction (i.e. reading and writing) with science inquiry (p. 7)

Literacy/Reading and Science Integration (p. 8)

The integration of reading instruction with science instruction is important in foster- ing scientific understanding for several reasons. First, reading becomes a window into the acquisition of scientific knowledge. (p. 8)

Second, reading is a process that complements scientific reasoning. (p. 8)

Third, reading is inevitably associated stu- dents’ ability to comprehend and succeed on high stakes assessments in science; stu- dents with lower reading abilities will have much more difficulties comprehending questions and therefore are more likely to perform poorly on those exams, including content area in science (Maerten-Rivera, Myers, Lee, & Penfield, 2010). Finally, science curriculum that integrates content learning and literacy development can promote reading skills and, not surprisingly, science content knowledge for low- SES diverse students (Connor et al., 2010; Fang & Wei, 2010). (p. 8)

More importantly, as was reported by Connor et al. (2010), students who had initially lower literacy skills made the same amount of gains as the students who had stronger literacy skills, given the explicit reading instruction integrated into the content instruction. (p. 8)

Hapgood and Palinscar (2007) proposed that science writing is an important approach, by which students ‘record questions of interest, document how they have set up investigations, represent data they have collected, and develop explanations for the phenomena they are investigating’ (p. 57). (p. 9)

The review of the literature revealed that the number of studies addressing literacy and science instructional integration at the middle school level for ethnically diverse, low-SES students is very limited (August, Branum-Martin, Cardenas-Hagan, & Francis, 2009; Author, 2012; Palumbo & Sanacore, 2009). Furthermore, from a methodological perspective, experimental and quasi-experimental designs with these students are rare; few studies are longitudinal that result in positive intervention effect on student achievement (August et al., 2009; Lee & Luykx, 2006). (p. 9)

Therefore, the purpose of our study was to evaluate the effectiveness of a literacy- integrated instructional intervention in academic science among Hispanic and African-American low-SES middle school students after one year of placement with comparison to the typical science instruction in the district (p. 9)

Specifically, we addressed the following research questions: (1) Do students’ performance differ on the district science benchmark tests by ethni- city (i.e. African-American vs. Hispanic), gender (male vs. female), and partici- pation in the literacy-embedded science intervention (i.e. treatment condition vs. comparison condition)? (2) Do students’ performance differ on the state-standardized science test by ethni- city, gender, and participation in the literacy-embedded science intervention? (3) Do students’ performance differ on the district reading benchmark tests and the state-standardized reading test by ethnicity, gender, and participation in the lit- eracy-embedded science intervention? (4) Do students’ performance differ on English oral reading fluency (ORF) by ethni- city, gender, and participation in the literacy-embedded science intervention? (p. 10)

Method (p. 10)

Context and Participants (p. 10)

Our study was part of a larger randomized, longitudinal, field-based research of lit- eracy-integrated science instruction delivered to ELL and non-ELL students from low-SES backgrounds (p. 10)

The district was chosen because of the district’s (a) ease of access to learning programs (i.e. regular and English-learning) within the same district, (b) lengthy experience working with low- SES students of color, © consistency in educational philosophy and implementation, and (d) reputation for student academic achievement as reflected by its national Broad Foundation Prize award. In this study, we selected participants who were His- panic or African-American and who spoke fluent English. (p. 10)

Design (p. 10)

4 schools with prin- cipals’ approval from the district site were randomly assigned to treatment (enhanced practice, n ¼ 2) and comparison (typical practice, n ¼ 2) conditions. (p. 10)

Both English language learners (ELLs) and low-SES non-ELLs in the same school received the same condition (enhanced or typical practice), so as to avoid contamination of the intervention between experimental and comparison classrooms. (p. 10)

reflects careful “control” (p. 10)

Teachers were ran- domly selected within the school for project participation, with four in treatment and eight in comparison schools (p. 10)

good practice talking about teachers (p. 10)

Therefore, the overall project was designed to be experimental at the school level, and quasi-experimental at the student level, abiding by the state law (Texas Education Code, 1995) prohibiting random selection and assignment on basis on individual students for program place- ment. (p. 11)

Axiology; ethics (p. 11)

only data collected from low-SES non-ELLs were included in the analysis, totaling 94 students from treatment schools (average age of 11.58 years, SD ¼ 0.75) and 194 comparison students (average age of 11.50 years, SD ¼ 0.71). (p. 11)

Literacy-Integrated Science Intervention (p. 11)

Teachers in the treatment condition were trained bi-weekly to assess their pedagogical progress and conduct experiments and inquiry activities in the overall intervention of Content Area Reading in Science for English Literacy and Language Acquisition (CRISELLA, see explanation below) (p. 11)

They also reviewed and practiced upcoming lessons based on the structured and scripted lesson plans that specifically listed science objectives, reading objectives, English-as-second-language (ESL) strategies, and target vocabulary for each week. (p. 11)

The lesson plan provided structure for teachers to model aca- demic science language, opportunities for students to practice using science academic language, and encouraged students to answer in complete sentences. (p. 11)

The science intervention was delivered in English for 85 minutes each day inclusive of the following major components: (a) Daily Oral and Written Language in Science (DOWLS) activity, in which students were presented with a science-related prompt and asked to think, record written responses, and discuss with a student partner and (b) CRISELLA, designed to improve students’ understanding of science (p. 11)

concepts through vocabulary development and extension in science-related exposi- tory texts. (p. 12)

Finally, in the third component, © Written and Academic oral language Vocabulary development in English in Science (WAVES), students recorded and illustrated a glossary of aca- demic science vocabulary in their individual science notebooks to help them process the content. (p. 12)

Typical Practice/Comparison Group (p. 12)

The science curriculum in the comparison classrooms is district-developed and aligned to the Texas Essential Knowledge and Skills (TEKS). (p. 12)

Fidelity (p. 13)

field notes from classroom observations were also collected from both treatment and control classrooms to monitor teaching behavior. (p. 13)

treatment teachers were able to effectively implement strategies (p. 13)

a positive effect of training workshops was evident because treat- ment teachers were making full use (p. 13)

treatment teachers from our bi-monthly meetings also reported that they have benefited from professional workshops (p. 13)

Measure (p. 13)

To evaluate the effect of this literacy-integrated science intervention, we used both standardized tests and district-developed tests in science and English language and lit- eracy, reflecting a multilevel, multifaceted assessment framework as suggested by Ruiz-Primo, Shavelson, Hamilton, and Klein (2002) in the assessment of science. (p. 13)

State-standardized test: Texas Assessment of Knowledge and Skills. The Texas Assess- ment of Knowledge and Skills (TAKS) is a state-standardized criterion-referenced assessment (p. 13)

District benchmark tests. Similar to TAKS, the district-developed benchmark test is also a criterion-referenced test to determine students’ performance in passing (p. 13)

Dynamic Indicators of Basic Literacy Skills. Dynamic Indicators of Basic Literacy Skills (DIBELS) (Good & Kaminski, 2002) measures early literacy skills from kinder- garten through sixth grade. (p. 14)

Data Sources and Analysis (p. 14)

Treatment effects were tested for vocabulary, fluency, and academic achievement in reading and science, respectively. (p. 14)

For ORF administered at the pre-test and post-test, analysis of covariance was used with the analogous pre-test serving as the covariate. (p. 14)

For measures for which the outcome variable is dichotomous, e.g. pass or fail in benchmark tests and TAKS tests, multilevel logistic analysis was performed to compare the odds ratio on these respective measures. (p. 14)

Considering the sample size of this study, cross- level interaction was not modeled, and the treatment effect was tested at the level of rotation/section. (p. 14)

possible dependency of students (level 1) within rotations (level 2) and within school (level 3) (p. 15)

Teacher was not included as a higher level because there were relatively small numbers of teachers in each school (close to three teachers per school, see Table 1), and school was the unit of randomization. Furthermore, adding both teacher and school levels to the model could have resulted in computational difficulty and a poten- tial non-convergence issue (p. 15)

Finally, the combined model follows: (p. 16)

All models were fit in HLM 6.08 (p. 16)

Results (p. 16)

Science Benchmark Tests (p. 16)

The first research question of this study was, Do students’ performance differ on the dis- trict science benchmark tests by ethnicity (i.e. African-American vs. Hispanic), gender (male vs. female), and participation in the literacy-embedded science intervention (i.e. treatment condition vs. comparison condition)? To respond to this research question, multilevel logistic analysis was conducted. (p. 16)

Using the stan- dard critical value of alpha ¼ 0.05 that is typical in educational research, the results in Tests 3 and 4 are considered statistically non-significant. However, given the limited literature, it is worthy of noting and future research (p. 18)

Furthermore, the treatment – gender – ethnicity difference was observed in the rate of commended performance in Test 4, with the coefficients of treatment, Female, and African-American estimated as significant with 1.23 (condition), 2 0.45 (gender), and 2 0.75 (ethnicity) (ps , .05), respectively, (p. 18)

Science TAKS Test (p. 18)

The second research question of this study was, Do students’ performance differ on the state-standardized science test by ethnicity, gender, and participation in the literacy-embedded science intervention? To respond to this research question, multilevel logistic analysis was conducted. (p. 18)

a marginally significant treat- ment effect (p ¼ .09), indicating that treatment students were 2.80 times (e1.03 ) more likely of passing the science TAKS test than their control peers. (p. 18)

Reading Achievement (p. 18)

The third research question of this study was, Do students’ performance differ on the dis- trict reading benchmark tests and state-standardized reading test by ethnicity, gender, and participation in the literacy-embedded science intervention? (p. 18)

Reading benchmark tests. (p. 20)

results indicated significant difference in Tests 2 and 4 in the rate of passing. (p. 20)

Reading TAKS test. (p. 21)

significant and negative ( r ¼ 2 1.12, p ¼ 0.001) in the rate of passing, corresponding to a higher chance of passing reading TAKS, i.e. 3.06 times more (e1.71 /e1.712 1.12 ) for Hispanic students than for African-American students across conditions and gender. (p. 21)

Oral Reading Fluency (p. 21)

The fourth research question of this study was, Do students’ performance differ on English ORF by ethnicity, gender, and participation in the literacy-embedded science inter- vention? (p. 21)

he coefficients of treat- ment, ethnicity, and pre-test were estimated to be statistically significant (ps , .01), indicating that controlling for pre-test scores, Hispanic male students in the treatment condition demonstrated 7.85 points higher than did their control peers, and Hispanic students scored 3.98 points higher than did African-American students on the ORF post-test. (p. 21)

Discussion and Recommendations (p. 21)

Using multilevel modeling techniques, we compared students’ performance by condition, gender, and ethnicity on academic achievement in reading and science and English reading fluency. (p. 21)

First, results suggest a positive treatment effect in the district-wide benchmark tests and the state-standardized test in science. More specifically, students who received the literacy-integrated science intervention were more likely than were the students in the typical practice (control) classrooms to pass the science benchmark tests (p. 21)

In addition to science achievement, we observed that treatment students also devel- oped faster in ORF. (p. 22)

A gender difference was found to be significant with male students demonstrating higher academic achievement in science than did female students. (p. 22)

It was also found that African-American students had a lower chance of sufficiently mastering the science concepts and of achieving above the state standards when com- pared with Hispanic students between genders and across conditions. (p. 23)

It is also legitimate to state that positive student outcomes are attributable to the bi- monthly professional development on reading/literacy-integrated, inquiry-based instruction with structured, standards-aligned lesson plans. (p. 23)

Significance, Implication, and Conclusions (p. 23)

opportunity for African-American and Hispanic students from low-income families to acquire more science knowledge and skills so as to potentially remove initial differences when they mature to eighth grade to compete with their middle-class White peers. (p. 24)

The second significance of our study is that our findings confirmed the positive effect of the literacy-integrated science intervention with English learners from the same larger research project (Author, 2012). (p. 24)

Implications related to professional development for teachers who teach low-SES students are for school administrators: (a) achieving similar outcomes on similar cam- puses, as reported in our study, requires administrators to provide a substantial amount of time and resource commitment for the teachers and (b) administrators should consider at least four hours of release time per month for teachers to develop skills outlined in our study that promote student learning in science and reading literacy. (p. 24)

Lee, O. (2005). Science education with English language learners: Synthesis and research agenda. Review of Educational Research, 75(4), 491 – 521. (p. 26)