Bodong Chen

Crisscross Landscapes

Notes: Strain - 2011 - Randomized, Controlled Trial of the LEAP Model of Early Intervention for Young Children With Autism Spectrum Disorders



Citekey: @Strain2011-sn

Strain, P. S., & Bovey, E. H. (2011). Randomized, Controlled Trial of the LEAP Model of Early Intervention for Young Children With Autism Spectrum Disorders. Topics in Early Childhood Special Education, 31(3), 133–154.



28 inclusive preschool classrooms were randomly assigned to receive 2 years of training and coaching to fidelity in the LEAP (Learning Experiences and Alternative Program for Preschoolers and Their Parents) preschool model, and 28 inclusive classes were assigned to receive intervention manuals only. (p. 1)

After 2 years, experimental class children were found to have made significantly greater improvement than their comparison cohorts on measures of cognitive, language, social, and problem behavior, and autism symptoms. Behavior at entry did not predict outcome nor did family socioeconomic status. The fidelity with which teachers implemented LEAP strategies did predict outcomes. (p. 1)

These one-to-one models of service delivery occasion the question as to whether educational resources are being used most effectively with young children with ASD. (p. 1)

In fact, of the 12 comprehensive programs identified by the National Research Council (2001), only LEAP and two other pro- grams rely on naturally occurring, incidental teaching arrangements. All other models rely on one-to-one, dis- crete-trial, or other adult-driven instructional approaches. Significantly, research evaluating the effects of one-to-one versus incidental tactics shows uniformly that the latter approach is either equal to or superior to one-to-one arrange- ments when generalized child gains are considered (Elliott, Hall, & Soper, 1991; McGee, Krantz, & McClannahan, 1985; Miranda-Linne & Melin, 1992). (p. 2)

However, after fundamental, nonoverlapping differences between groups were evident after 2 years, a NIMH site visit panel recommended suspension of random assignment for ethical reasons. We fully concurred with this recommendation not- withstanding the obvious compromise in the study design. (p. 2)

Ethics. (p. 2)

Second, we have considered it to be ethically and scien- tifically necessary to conduct various substudies to demon- strate the efficacy of key model components that are considered to be unique LEAP features. (p. 2)

The overall study had five primary research questions: Research Question 1: Do differential child outcomes occur across study groups after 2 years of LEAP implementation? Research Question 2: What is the relationship between implementation fidelity and child outcomes? Research Question 3: Are teacher’s characteristics correlated with child outcomes? Research Question 4: Does child performance at the beginning of intervention predict ultimate out- comes at 2 years? Research Question 5: How do teachers rate the social validity of their experience implementing LEAP? (p. 3)

Third, in spite of these prior data, we determined that the overall efficacy of enrollment in LEAP could not be argued without a randomized trial. This RCT was funded by the Institute for Education Sciences (IES) under the “efficacy and replication” goal. To qualify for support under this com- petition, several requirements must be met, including (a) the intervention must be fully developed and to the extent pos- sible, manualized; (b) prior data must provide a convincing case about potential efficacy; and © the intervention must be implemented in authentic educational settings by typical intervention agents. (p. 3)

Method (p. 3)

Overall LEAP Description (p. 3)

The LEAP model has a number of unique features that contribute to its effectiveness and relatively lower costs. (p. 3)

Although single-case designs on children with ASD have long utilized fidelity of treatment measures, such has not occurred in RCTs of com- prehensive treatments. (p. 3)

occurred at the classroom level, it followed logically to randomly assign classrooms. (p. 4)

Thus, using our treatment manuals without follow-along training was a logical standard for comparison purposes. (p. 4)

Randomization in study group status along with equating preschools on dimensions of adult–child ratio (1:5), full inclusion (all ser- vices received in classrooms), and intensity of services (17 hr/ week on average) offered the best available analytic model. (p. 4)

Participant Selection (p. 4)

Overall Study Design (p. 4)

we “overre- cruited” classrooms from school district settings that were willing and able to be LEAP replication sites (p. 4)

We instituted a clustered randomized comparison design in which preschool classrooms matched on program dimen- sions such as number of program days per week (5) and length of program day (2.75–3 hr) and were assigned, via a table of random numbers, to either the full-scale LEAP replication training or to a comparison condition in which preschool staff were provided intervention manuals and related written materials only. (p. 4)

Table 1. Key Input Differences Between LEAP Replication Classes and Manuals-Only Classes (p. 6)

Experimental Conditions (p. 6)

Preschools assigned to the comparison condition were provided LEAP’s intervention manuals, videos, and train- ing presentation materials (Power Point presentations) for (a) family skill training, (b) social skills training, and © design and operation of the inclusive classroom. (p. 6)

Fidelity of Intervention Concerns For 14 years, we have used an implementation protocol in the form of a procedural rating scale (QPI). The QPI con- sists of eight content areas (Classroom Organization and Planning, Teaching Strategies, Teaching Communication Skills, Promoting Social Interactions, Providing Positive Behavioral Guidance, IEPs and Measuring Progress/Data Collection, and Interactions With Children and Interactions With Families) with each content area having four to six unique indicators. (p. 6)

Mullen Scales of Early Learning. This developmental scale provides an assessment of child performance in visual reception, fine motor, receptive language, expressive lan- guage, and a composite representing general intelligence. The Mullen (Mullen, 1995) has excellent internal reliabil- ity (.91) and short-term test–retest reliability (.95). (p. 7)

Preschool Language Scale (4th ed.; PLS-4). The PLS-4 (Zimmerman, Steiner, & Pond, 2002) provides a compre- hensive assessment of children’s receptive and expressive communication competence. (p. 7)

Child Measures and Measurement Methods (p. 7)

We designed the measurement protocol with the following considerations in mind: (a) We wanted to compare child outcomes at replication sites with our initial comparison intervention study of LEAP, (b) we wanted to address the defining characteristics of autism (i.e., language devia- tions, social isolation, overall developmental delay, and various forms of aberrant behavior), © we wanted to select measures that have demonstrated sensitivity to simi- lar interventions, and (d) we wanted to use measures that provide reliable and valid data. (p. 7)

Social Skills Rating System (SSRS). The teacher form of the SSRS (Gresham & Elliott, 1990) was used to assess changes in child participants’ social skill development and problem behavior. (p. 7)

Childhood Autism Rating Scale (CARS). (p. 7)

Social validity measure. At the end of our 2-year consul- tation process, the lead teacher in each intervention class was asked to complete a 5-point rating scale (see Appen- dix C) that was designed to assess consumers’ judgments on key dimensions of their consultation relationship with LEAP coaches. The 14 dimensions of social validity were based on Kohler and Strain’s (1992) review of practice dimensions shown to influence long-term use of evidence- based practices. (p. 7)

Table 3. Study Participants’ Data For Full LEAP Replication and Manuals-Only Groups at Start (p. 8)

Child Outcomes (p. 8)

Results (p. 8)

Table 4. Child Outcomes After 2 Years of Study Participation (p. 9)

t is also true that the data set would permit growth curve analyses; how- ever, our theory of change ran counter to this analytic method. (p. 9)

Multivariate repeated-measures analysis of variance was performed on clusters of classrooms, not individual child data. All differences between groups were significant at the .05 level and beyond. This analytic method was cho- sen because we had (a) no “missing data”, (b) the assess- ment schedule was consistent across all participants, © this analytic method was appropriate to the basic study design, and (d) multivariate repeated-measures analysis of variance has been used in all prior early childhood autism RCTs. (p. 9)

Social Validity Ratings (p. 9)

Although hierarchical linear modeling was a theo- retically plausible analytic method, power was not found (p. 9)

Table 5. Correlation Between QPI Scores at End of Year 2 and Each Outcome Index Gain Score for Full Replication and Manuals-Only Classes (p. 10)

Methods used to mitigate bias in the data gathering and analytic processes included (a) keeping assessors naïve to study hypotheses; (b) having a personnel “firewall” separat- ing staff who were working with sites to reach fidelity and those collecting outcome data; © having data collection, scoring, and storage subject to multiple reliability/accuracy checks; and (d) having outcome data processed independent of the principal investigator. (p. 10)

Good practice to explain this (p. 10)

Discussion (p. 10)

The results of this RCT on young children with ASD revealed the following results: (a) Fidelity of implemen- tation data showed that intervention classes reached high levels of fidelity (nearly 90% of practices in place) after 2 years of coaching, whereas comparison classes using the same manualized materials independent of coaching were implementing 38% of LEAP practices after 2 years; (b) children in intervention classes made significantly more progress than comparison children at the end of 2 years on measures of cognitive, language, autism symptoms, prob- lem behavior, and social skills; © differential child out- comes were not correlated with initial baseline performance, lead teachers’ level of experience or prior training, or with family socioeconomic status; (d) significant correlations were found between fidelity scores in intervention and comparison classes and all child outcomes; and (e) the pri- mary teachers in intervention classes rated their experience implementing LEAP practices very high, and they judged child behavior change to be attributable to these practices. (p. 10)

As seen in Table 6, teacher consumers had very favor- able ratings of their experience with the LEAP replication process. For the subgroup of teachers in the intervention cohorts, we examined the relationship between social validity ratings and fidelity scores at the end of coaching. The correlation between these measures was highly signifi- cant (r = .89). (p. 10)