Increasing functional communication skills in childen with autism: a meta-analysis

A plethora of research exists regarding various teaching techniques and interventions for autism. By consolidating and analyzing the available research, effective interventions may be identified which can help lead to better practices. Parents, teachers, clinicians, and care providers, in addition to individuals with autism, stand to benefit from a systematic evaluation of the treatment modalities available for increasing functional communication skills in children with autism. A statistical analysis was performed on the available data using the computer program, Hierarchical Linear Modeling (HLM), which allowed for analyses of participants nested within studies. Effect sizes (ES) were calculated for PECS, Sign Only Communication and Total Communication interventions described in the research. This common metric allowed for comparisons to be made between treatments while taking individual differences into account. This meta-analysis included 43 single-subject outcome studies completed between 1965 and 2004. Quantitative results indicated that total communication (TC) interventions resulted in the largest mean effect sizes. Additionally, several participant characteristics (e.g., adaptive score, age, IQ, receptive language score, verbal/nonverbal classification) and one study characteristic (e.g., data base) were found to moderate treatment effectiveness. The practical implications from these results are vast. First and foremost, regardless of personal preference, it is critical that practitioners understand that TC interventions always resulted in the greatest treatment gains, making it difficult to suppose a situation that either SIGN or PECS be implemented over TC. Second, even though individuals with autism can be difficult to accurately assess using standardized measures, it is critical that practitioners and researchers attempt to get accurate pretreatment scores being that many are highly predictive of individual outcomes. Finally, the results help lay a foundation for future research aimed at comparing topography-based and selection-based communication interventions.

INCREASING FUNCTIONAL COMMUNICATION SKILLS IN CHILDREN WITH AUTISM: A META-ANALYSIS by Lisa Marie Goldy A thesis submitted to the faculty of The University of Utah in partial fulfillment of the requirements for the degree of Master of Science Department of Educational Psychology The University of Utah May 2009 Copyright © Lisa Marie Goldy 2009 All Rights Reserved THE U N I V E R S I T Y OF UTAH G R A D U A T E SCHOOL SUPERVISORY COMMITTEE APPROVAL of a thesis submitted by Lisa Marie Goldy This thesis has been read by each member of the following supervisory committee and by majority vote has been found to be satisfactory. John C. Kircher UNIVERSITY GRADUAT E SCHOOL Lora Tuesday-Heath wId THE U N I V E R S I T Y OF UTAH G R A D U A T E SCHOOL APPROVAL To the Graduate Council of the University of Utah: I have read the thesis of Lisa Marie Goldy m j t s final form and have found that (1) its format, citations, and bibliographic style are consistent and acceptable; (2) its illustrative materials including figures, tables, and charts are in place; and (3) the final manuscript is satisfactory to the supervisory committee and is ready for submission to The Graduate School. Wflliam R. Jenson .hair: Supervisory Committee Approved for the Major Department ^ Elaine Clark Chair/Dean Approved for the Graduate Council David S. Chapman(| Dean of The Graduate School UNIVERSITY GRADUATE SCHOOL FINAL READING APPROVAL . in its final Date Chapman ABSTRACT A plethora of research exists regarding various teaching techniques and interventions for autism. By consolidating and analyzing the available research, effective interventions may be identified which can help lead to better practices. Parents, teachers, clinicians, and care providers, in addition to individuals with autism, stand to benefit from a systematic evaluation of the treatment modalities available for increasing functional communication skills in children with autism. A statistical analysis was performed on the available data using the computer program, Hierarchical Linear Modeling (HLM), which allowed for analyses of participants nested within studies. Effect sizes (ES) were calculated for PECS, Sign Only Communication and Total Communication interventions described in the research. This common metric allowed for comparisons to be made between treatments while taking individual differences into account. This meta-analysis included 43 single-subject outcome studies completed between 1965 and 2004. Quantitative results indicated that total communication (TC) interventions resulted in the largest mean effect sizes. Additionally, several participant characteristics (e.g., adaptive score, age, IQ, receptive language score, verbal/nonverbal classification) and one study characteristic (e.g., data base) were found to moderate treatment effectiveness. The practical implications from these results are vast. First and foremost, regardless of personal preference, it is critical that practitioners understand that TC effective effectiveness. interventions always resulted in the greatest treatment gains, making it difficult to suppose a situation that either SIGN or PECS be implemented over TC. Second, even though individuals with autism can be difficult to accurately assess using standardized measures, it is critical that practitioners and researchers attempt to get accurate pretreatment scores being that many are highly predictive of individual outcomes. Finally, the results help lay a foundation for future research aimed at comparing topography-based and selection-based communication interventions. v PEeS v To all the children who have taught me so much; and, especially to my daughter, Ellie, who teaches me each day how much more I have to learn. TABLE OF CONTENTS Page II. METHOD 35 Literature Search Procedure 35 Criteria for Study Inclusion 36 Method for Analyzing Research Articles 40 Characteristics of the Study 47 Examination of Research Questions 51 Discussion 83 Implications for Practice 92 Limitations 94 Future Research 96 ABSTRACT .................................................................................................................. .iv LIST OF TABLES ........................................................................................................ .ix LIST OF FIGURES ....................................................................................................... xi CHAPTER I. INTRODUCTION ................................................................... ....... .................... 1 Autism Spectrum Disorders ............................................................................... 1 Language Interventions for Individuals with ASDs ........................................... 6 Study Rationale ................................................................................................. 25 Meta-Analysis ................................................................................................... 25 Purpose .......................................................................................................... .... 31 Research Questions ...... ...... ..... .......................................................................... 33 .................................................. ....................................................... Design .. ......... ................................. ..... .......................................................... .... 35 .............................................................................. ........ ......................................................... ............. ......................................................... III. RESULTS ......................................................................................................... 47 .............................................................................. Reliability .......................................................................................................... 51 ................................................................. IV. DISCUSSION ................................................................................................... 83 ........................................................................................................ ................................................................................... ........................................................................................................ 94 ....... .................................... ................... ...... ............................ 96 Conclusion 97 A. CODING SHEET 98 B. INDIVIDUAL DATA POINTS CODING SHEET 101 C. HLM FINAL ESTIMATION OF VARIANCE COMPONENTS 103 D. FREQUENCIES FOR JOURNAL BY CATEGORY 112 REFERENCES 114 viii .......... ............. ... .... ........ ..... ....... ...... .... ....... ...................... ... ............ APPENDICES ...................... ................ ........................................................ ........................................ I0l .................. .......................... ......... ........................................................................................................... V111 LIST OF TABLES Table Page 1. Reasons for Excluding Studies From Meta-Analysis 48 2. Characteristics of Included Studies 49 3. Characteristics of Participants Included 52 59 17. Pair-wise Comparisons by Journal 75 18. Pair-wise Comparisons by Research Design 76 ...................................... .48 .................................................................. .49 ............................................................ 4. Summary Statistics for Moderator Variables .................................................... 53 5. Reliability .......................................................................................................... 53 6. Composite Effect Size by Treatment Type ....................................................... 54 7. Pair-wise Comparisons for Each Treatment Type ............................................ 55 8. Frequencies for Treatment by Measurement Type ........................................... 55 9. Pair-wise Comparisons With Measurement Type Controlled .......................... 56 10. Frequencies by Treatment and Language Type ............................................... .59 11. Mean Effect Size With Language Type Controlled .......................................... 59 12. Pair-wise Comparisons by Treatment for Each Language Type ...................... 61 13. Pair-wise Comparisons by Diagnosis ............................................................... 63 14. Pair-wise Comparisons by Cognitive Classification ......................................... 64 15. Pair-wise Comparisons by Verbal Classification ............................................. 66 16. Pair-wise Comparisons by Data Base ............................................................... 73 Joumal .................................................................... .................................................... 20. Pair-wise Comparisons by Treatment Implementer 79 21. Pair-wise Comparisons by Treatment Setting 80 23. Final Estimation of Fully Unconstrained Variance Components 104 24. Final Estimation of Treatment Type Variance Components 104 25. Final Estimation of Language Type Variance Components 104 26. Final Estimation of Treatment X Language Type Variance Components 104 27. Final Estimation of Participant Level Variance Components 105 28. Final Estimation of Treatment X Participant Variance Components 107 Ill 19. Pair-wise Comparisons by Generalization Type ............................................... 77 ......................................... ................................................... 22. Fail-Safe N by Effect Size Estimate ................................................................. 82 ............... ..................... ..................... ..... ................. ,. ........... 29. Final Estimation of Study-Level Variance Components ....... , ................ 108 30. Final Estimation of Treatment X Study-Level Variance Components ......... 111 31. Frequencies for Journal by Category ................................................. 113 LIST OF FIGURES Figure Page 1. Mean Effect Size by Treatment and Measurement Type 56 2. Mean Effect Size by Language Type 58 3. Treatment X Language Type Interaction 60 4. Mean Effect Size by Verbal Classification 67 5. Mean Effect Size by Nonverbal and Verbal Classifications 67 6. Effect of Duration and Maturation on Global Effect Size 69 7. Adaptive S core X Treatment Interaction 71 8. IQ X Treatment Interaction 72 9. Mean Effect Size by Data Base 74 ~~re p~ .................................. ............................. ................................... ...... .................................................... ........................ ............................... ............................. ................................ Score ......................... ................................. ............ ... .... .... ... ....... ............. .......... ........ .... .... ....... ............................................................. ........... CHAPTER I INTRODUCTION Autism Spectrum Disorders Autism is one of a group of disorders known as autism spectrum disorders (ASDs). ASDs are developmental disabilities and include autistic disorder, pervasive developmental disorder - not otherwise specified (PDD-NOS) and Asperger syndrome. Autism is the most common pervasive developmental disorder (National Institute on Deafness and Other Communication Disorders (NIDCD), 2002). Due to the variability and severity of symptoms that many children with ASDs endorse, education, psychology, neurology and the medical professions in general are continually seeking new interventions and treatments that will help mitigate some of the devastating effects autism may have on individuals, families and communities. Diagnostic impressions typically include delays in language development, delays in understanding social relationships, inconsistent patterns of sensory response, uneven patterns of cognitive functioning and marked restriction of activities and interests (American Psychiatric Association, 2001). Many individuals with autism engage in self-injurious and aggressive behaviors requiring more restrictive educational settings as well as limited community involvement. Tager- Flusberg (1999) emphasizes that, "Research on autism and other neurologic disorders suggests that the social and communication impairments are unique and specific deficits that define the autism phenotype" (p. 325). Tager­Flusberg 2 Prevalence of Autism Spectrum Disorders "While the cause remains a mystery, most specialists now view autism as a brain disorder that makes it difficult for the person to process and respond to the world" (NIDCD, 2002, p.l). "For decades autism was believed to occur in 4 to 5 per 10,000 children" (U.S. Centers for Disease and Prevention, 2007, p. 3). The average prevalence for ASDs in the United States is currently one in every 150 children (CDC, 2007). Incidence is difficult to determine because it is not known when an individual develops an ASD; however, estimates using current prevalence and birth rates suggest that up to 560,000 individuals between the ages of 0 and 21 have an ASD. Data from the Individuals with Disabilities Education Act (IDEA) show that 30,305 3- to 5-year-olds and 193,637 6- to 21-year-olds received public special education services under an "autism" classification in 2005. Further, of the more than 6,000,000 students receiving special education services in 2005, autism made up approximately 3% (Individuals with Disabilities Education Act Data, 2006). Although the knowledge regarding this disorder is increasing, there is no known cure for autism. There are, however, recommendations for effective treatment for children with autism as outlined by The National Research Council (NRC). The committee found that, "Functional, spontaneous communication should be the primary focus of early education. For very young children, programming should be based on the assumption that most children can learn to speak. Effective teaching techniques for both verbal language and alternative modes of functional communication, drawn from the empirical and theoretical literature, should be vigorously applied across settings" (National Research Council, 2001, p.221). Miller (2003) and 1). 193,6376- Effective applied across settings" (National Research Council, 2001, p.221). Miller (2003) and Grandin and Scariano (2005) note that one of the primary indicators of good prognosis is meaningful speech by age 5. Language Deficits and Autism Spectrum Disorders Language by definition refers to the formal linguistic code that we acquire as our primary means for communication, including speech, sounds, meaning and grammatical components. Language is the means for interacting with others and for sharing information, thoughts or feelings between a "speaker" and "listener." With typically developing children, the pattern of communication begins in infancy with attention to human faces and voices. Infants engage in eye contact with other people and respond to affective expression in both faces and voices. By age 2, interactions with others center on directing attention to themselves or to people or objects in the environment. This is done by vocal and gestural means with knowledge of a variety of single words. "During this period children rapidly learn that words symbolize or represent objects, actions, and thoughts" (NIDCD, 2002, p.2). Toddlers and preschool-age children typically engage in social communication - the focus is on play, beginning with simple imitation of actions then moving toward incorporating pretend or imaginative play and activities. By age 3, most children are able to communicate in short phrases. Pragmatics by definition is the ability to use language appropriately in social contexts (National Institute of Mental Health (NIMH), 2003; Tager-Flusberg, 1999). Unfortunately, children with autism are diagnosed by specific deficits in many of these areas. Although not comprehensive, autistic disorder is laid out in the fourth edition of the Diagnostic and Statistical Manual (DSM-IV-TR) as including impairments in the following areas: 3 ofthe ofthese Manual 4 299.00 Autistic Disorder Al Qualitative Impairment in Social Interaction a. marked impairment in the use of multiple nonverbal behaviors: eye-to-eye gaze, facial expression, body postures & gestures to regulate social interaction c. lack of spontaneous seeking to share enjoyment, interests or achievements with other people (e.g., by a lack of showing, bringing, or pointing out objects of interest) A2 Qualitative Impairments in Communication a. delay in or total lack of the development of spoken language (not accompanied by an attempt to compensate through alternative modes of communication such a gesture or mimes) b. in individuals with adequate speech, marked impairment in the ability to initiate or sustain conversation with others c. stereotyped and repetitive use of language or idiosyncratic language B Delays or abnormal functioning in at least one of the following areas, with onset prior to age 3 years: (1) social interaction, (2) language as used in social communication, or (3) symbolic or imaginative play, (pp.70-75) Language, or more specifically functional communication, allows individuals to get their needs met, to express themselves and to understand and respond to others. Sattler (2002) states, "Listening and speaking are the first language processes to develop and remain the most competent through the upper elementary years" (p.632). Language of children with typical development. has been estimated that 50% of all individuals diagnosed with autism will remain nonverbal (Prizant, 1983), and unfortunately as Sattler (2002) notes, reading and writing processes depend on the depth and breadth of previously acquired listening and speaking abilities. Children with autism do not necessarily use speech for communicative purposes. "Instead their speech is typified by echolalia - the repetition of words and phrases, a failure to use personal pronouns and a lack of inflection, making their speech sound emotionless. These children also have a play. acquisition is developmental in nature; therefore, gains are anticipated over time. Since autism is a developmental disorder, children's skill acquisition does not mirror that It 5 limited comprehension of the speech of others, making them appear to be noncooperative" (Koegel, Rincover, & Egel, 1982, p.85). Fortunately, understanding and initiating auditory speech is only one means by which individuals are able to successfully interact with others; however, "They (individuals with autism) are often unable to use gestures either as a primary means of communication, as in sign language, or to assist verbal communication, such as pointing to an object they want" (NIDCD, 2002, p.2). For purposes of this study, language and functional communication will be used interchangeably and will be defined as the following: Language is receptive and expressive, verbal and nonverbal and includes the functional exchange of symbols between two individuals resulting in behavior(s) that indicate comprehension of the exchange. Language will not be defined as the topography of speech sounds, intonation, articulation, etc. Historically researchers have found that children with autism may benefit from intensive behavioral treatment, particularly if the children are younger than 46 months at the start of the intervention (Lovaas, 1987). "Although children with autistic spectrum disorders share some characteristics with children who have other developmental disorders and may benefit from many of the same educational techniques, they offer unique challenges to families, teachers, and others who work with them. Their deficits in nonverbal and verbal communication require intense effort and skill even in the teaching of basic information" (NRC, 2001, p.213). When Scruggs and colleagues analyzed 20 single-subject investigations focused on early language interventions for preschoolers with language disorders or deficits, they found, ".. .that the type of dependent variable was a stronger predictor of outcome than was the training procedure" (Scruggs, successfully ofthis ofthe " ... 6 Mastropieri, Forness, & Kavale, 1988, p. 271). They also found that outcomes specific to language acquisition were higher than outcomes associated with increased language use and that, ".. .generalization effects, taken as a whole, were lower than training effects" (Scruggs et al., 1988, p.280). Language Interventions for Individuals with ASDs Although many interventions that are used for children with autism are meant to be all encompassing, specific interventions or intervention components may result in increased language skills. In her commentary, Achievements and future directions for intervention research in communication and autism spectrum disorders, Lord (2000) highlights, "There is a substantial literature that shows that treatments that specifically target particular communicative features can effectively teach these targets: there is a smaller literature that shows that comprehensive educational programs that contain communication goals result in general gains in language skills" (p. 394). Goldstein (2002) conducted an extensive review of communication interventions for children with autism. He found enough evidence to claim that effective interventions do exist to teach communication skills to children with autism, but the literature does not provide adequate information regarding appropriate service delivery models or intensity of services needed to maximize the benefits. Many autism interventions have overlapping characteristics; however, in order to summarize the interventions geared at increasing functional communication skills in children and young adults with autism, the treatment methods can be grouped by theoretical or treatment similarities. Although not comprehensive, the following categories provide an overview of the more prevalent and/or current " ... aI., for 7 interventions being utilized to help improve communication skills for individuals with autism spectrum disorders. Applied Behavior Analysis (ABA) Applied Behavior Analysis (ABA) is a theory of behavior modification based on the premise that behavior rewarded is more likely to be repeated than behavior ignored. Although ABA is a theory, it is often used as a term to describe a specific treatment approach (Autism Society of America (ASA), 2002). Additionally, in the field of autism, the term ABA is often used interchangeably with "Lovaas" and "Discrete Trial Training"; however, ABA is actually a behavior theory that is based on three fundamental assumptions. First, any behavior has an antecedent, followed by the behavior and a consequence (ABC). Second, the behavior is related to the antecedent and past consequences. Third, effective teaching involves control over the antecedents and consequences. In addition to antecedents, behaviors and consequences, reinforcement and punishment are key components of applied behavior analysis. Teaching strategies include a variety of behavioral techniques such as prompting, modeling, fading, errorless learning, chaining (both forward and backward), task analysis, shaping, generalization and maintenance. Discrete Trial Format is a useful teaching format in which information is presented in a specific way with a clear beginning and a clear ending. Discrete Trial (DT) Training is an intensive approach to teaching new skills in which ".. .every task given to the child consists of a request to perform a specific action, a response from the child, and a reaction from the therapist. Tasks are broken down into short simple pieces fundamental which" ... 8 or trials. When a task has been successfully completed, a reward is offered, reinforcing the behavior or task" (ASA, 2002, p.2). Parent-Directed or Managed Intensive Behavioral Therapy is similar to Lovaas discrete trial training but due to a shortage of trained therapists, parents manage the in-home intervention (Sheinkopf, Gables, & Siegel, 1998). According to Michalski (2003), the ingredients for a successful home-based behavior intervention for children with autism include early intervention between 2 and 5 years of age, high intensity intervention that occurs year round between 35 and 40 hours per week and involves parents. With the goal of school integration, the foundation behaviors include reducing negative behaviors such as self-stimulatory behaviors, tantrums and general noncompliance; reducing self-injurious behaviors; increasing positive behaviors such as language skills (e.g., length of utterance, early conversation skills), abstract concepts (e.g., object labeling, shapes, colors, counting), age-appropriate toy play skills (imitative, imaginative, and interactive) and self-help skills (e.g., potty training). Outcome measures include increases in IQ scores, but controversies include reliance on aversives, nonrandom assignment and lack of group matching (Miller, 2003). Newer research (Smith, Groen, & Wynn, 2000) attempts to overcome some of the criticisms resulting in treatment effects that are roughly halved. Intensive Behavior Intervention (IBI) or Lovaas Therapy is behavior modification that uses the ABC model and advocates intervention at an early age with Lovaas-trained practitioners typically providing a minimum of 20 to 40 hours per week of one-on-one behavioral therapy in the child's home for a minimum of 12 to 24 months with the goal of achieving kindergarten readiness by focusing on developing skills in attending, in­home obj ect ofthe modification of20 imitation, both receptive and expressive language, preacademics and self-help (ASA, 2002; Davis, Smith, & Donahoe, 2002; Lovaas, 1981, 1993). The UCLA Young Autism Project has evolved over time. Initially children were removed from the home and received one-on-one behavioral training during most waking hours. Although a moderate number of studies utilized operant conditioning to teach speech to nonverbal children with autism, the progress was slow (Lovaas, Berberich, Perloff, & Schaeffer, 1966). When data revealed that gains were not maintained and did not generalize after discharge to the home, the intervention became home-based. The intensity of the one-on-one behavioral training changed to 40 hours per week for a 2-year duration. This behavioral approach is based on operant conditioning principles and utilizes discrete trials to achieve errorless learning. As previously stated, these techniques lend themselves well to data collection; publications abound, resulting in Lovaas being the most research-supported intervention for comprehensively treating autism. Unfortunately the results of operant speech and language training are not uniformly positive. Some children require thousands of trials before mastering a simple discrimination (Lovaas, Schreibman, Koegel, & Rehm, 1971). Verbal Only Communication (VQ Verbal Only Communication (VC) can be defined as oral training of speech using only vocal cues. This operant language training technique focuses on verbal speech rather than less functional forms of communication such as sign language. The initial goal of training is verbal imitation with a gradual fading of verbal prompts. For verbal children with autism, many therapists feel this should be the first choice of language interventions, which is consistent with the NRC (2001) findings that functional, 9 Proj ect (VC) functional, 10 spontaneous communication should be the primary focus of early education for children with autism. And, for very young children, programming should be based on the assumption that most children can learn to speak. Delprato (2001) ".. .examined a series of 10 controlled studies in which traditional operant behavioral procedures were compared with more recently developed normalized interventions for teaching language to young children with autism" (p. 315). In normalized interventions natural versus arbitrary reinforcers are utilized. Traditionally a child may be reinforced or rewarded for a correct response with an arbitrary sip of juice - regardless of the correct response. In contrast, a natural reinforcer serves as a reinforcer because it is directly related to the requested response. "Reinforcing a child with a sip of juice following the to-be-learned spontaneous verbalization, 'I want juice' is more natural than making a piece of cookie contingent on the desired response" (p. 318). Delprato summarized that in controlled comparisons of children with autism, normalized teaching was consistently superior to discrete-trial training. According to the University of North Carolina, Chapel Hill website (2002), current problems with language interventions for children with autism lie in the overall ineffectiveness of teaching spontaneous verbal initiations as well as the multifunctions of language. Spontaneous initiations are crucial for social interactions and are generally used as an indicator of social competence (Koegel, 2000). Some recent findings have been promising in effectively improving verbal initiations, such as question asking (Koegel, Camarata, Valdez-Meachaca, & Koegel, 1998). However, Wetherby and Prutting (1984) analyzed language samples of verbal children with autism and found them deficient quantitatively and qualitatively in functions of language; language seemed " ... Iearned oflanguage; 11 to be used primarily for requesting objects, requesting actions, and protesting. Teaching children additional uses of language is imperative for future interventions geared at increasing functional communication skills. Unaided Augmentative and Alternative Communication (AAC) "Many individuals with autism are candidates for augmentative and alternative communication (AAC) systems, either to supplement (i.e., augment) their existing speech or to act as their primary (i.e., alternative) method of expressive communication" (Mirenda, 2003, p. 203). In his narrative review of functional augmentative and alternative communication aids for children with autism, Mirenda (2003) simply states that there are two basic types of AAC: unaided and aided. "Unaided communication does not require any equipment that is external to the body and involves the use of symbols such as manual signs, pantomimes, and gestures" (p. 204). Total Communication (TO Total Communication or simultaneous communication can be defined as pairing sign language with verbal communication. Oxman, Webster, and Konstantareas (1978) note that the primary emphasis of simultaneous communication is the manual component; however, improvement in speech has also been cited as a "coincidental" benefit. Since many nonverbal children with autism appear to have difficulty understanding spoken language, signing, a more visual, gestural form of communication may be more accessible to these children. Behavioral techniques are most often used to teach this unaided alternative form of communication. Parents and teachers can mold or manually prompt a child's hands into appropriate sign configurations, and many hope that verbal communication (TC) 12 pairing will trigger or facilitate speech development (Creedon, 1973; Goldstein, 2002). Additionally, Koegel et al. (1982) point out that for many signs there is a concrete relationship between the sign and its referent. Similarly, verbal communication can also be paired with pictures or symbols in the hopes of increasing functional communication skills. Acquisition time is thought to be reduced when using concrete visual stimuli with the more abstract verbal communication. Yoder and Layton (1988) found that verbal imitation skills accounted for the majority of variation between participants who did and did not demonstrate spoken language after total communication instruction. Barrera and Sulzer-Azaroff (1983) cite several researchers who have found total communication training to be effective in teaching basic expressive language skills to mute autistic children. In spite of these results, they note that many therapists avoid training in less functional sign language, and they address the phenomenon known as stimulus overselectivity and its disputed influence on total communication. When simultaneously presented two or more stimuli, many children with autism are only able to signs and spoken words are presented simultaneously, Dores and Carr (1979) found a positive correlation between a child's score on a pretraining test of verbal imitation and the ability to respond correctly to posttraining auditory probes, independent of chronological or mental age. Some researchers have found through stimulus control analyses, that it is a child's verbal imitation skills that predict whether he/she will experience improvements in receptive speech and/or receptive signing after total communication training. Others believe that overselective responding is related to mental age and is less pronounced in higher functioning individuals with autism. did not demonstrate spoken language after total communication instruction. Azaroff(attend to one of the stimuli. In an attempt to systematically assess what is learned when 13 Goldstein (2002) comments in his review of communication interventions for children with autism that interest in TC has waned even though, "Total communication appears to be a viable treatment strategy for teaching receptive and expressive vocabulary (i.e., language content) to individuals with autism" (p. 385). In a review of behavioral treatments, Matson and colleagues note the majority of studies they reviewed evaluated techniques for increasing expressive rather than receptive language, and many of the researchers attempted to address whether verbal, nonverbal or total communication is the appropriate language modality for individuals with autism (Matson, Benavidez, Compton, Paclawskyj, & Baglio, 1996). Matson et al. (1996) cite, "Newsome and Rincover (1989) recommend teaching language using the total communication approach when it is unclear as to which modality is most appropriate" (p. 446). Sign Only Communication (SIGN) Sign only communication (SIGN) is categorized as an unaided alternative form of communication in which operant conditioning has been used extensively in the training process. The vast majority of studies involve complex treatment packages making it difficult to attribute gains to any one intervention component. Consequently, these multifaceted treatments are difficult at best to replicate. In a review of sign language or simultaneous sign and speech communication intervention studies, "Data on over 100 children indicate that nearly all autistic children learn receptive and expressive signs, and many learn to combine signs" (Bonvillian, Nelson, & Rhyne, 1981, p. 125). They found that two of the more commonly used sign language systems were American Sign Language (ASL) and Signing Exact English (SEE). A third sign language system found anyone 14 in the literature is known as Ontario Sign Language (Konstantareas, Oxman, & Webster, 1977; Konstantareas, Webster, & Oxman, 1979). Unfortunately, sign training has mixed results in the literature. ".. .Not all children with autism perform equally well with regard to manual sign learning, and one variable that appears to be related to outcome is fine motor ability" (Mirenda, 2003, p. 204). However, "Overall the level and rate of mastery for subjects learning signs appears to be greater than that for speech in comparable speech-oriented operant programs" (Bonvillian et a l , 1981, p. 126). Creedon (1973) emphasizes that the initial rationale for teaching sign language to nonverbal children with autism was behaviorally based. A more efficient form of communication instruction was necessary with this population due to the ineffectiveness of the well-documented verbal communication training. In her ground-breaking study, Creedon (1973) stresses that there is "... a need for the most expedient form of communication training, a form that could be shaped from existing behaviors (hand posturing), and a form that provided many opportunities for consequences to be attached" (p. 1). Although Creedon (1973) employed a simultaneous approach to teaching communication, she notes that despite the variable acquisition rates, all 21 children who participated in the process followed the same sequence of learning. "In the early stages of learning signs, the children were echopraxic. That is, they imitated all the hand movements of the teacher" (p.3). Verbal communication did not necessarily ensue, but all children were able to get their immediate needs met by utilizing manual sign language. Parent participation was crucial; when parents are taught to sign, there are clearer increases in communication. " ... aI., ofthe " ... 15 Although often argued to be a less functional form of communication, as stated previously it has been estimated that as many as 50% of children with autism will remain functionally nonverbal throughout their lifetimes making sign language appealing as a means of increasing functional communication skills as long as communication partners are familiar with manual signs. In a review of functional augmentative and alternative communication (AAC) interventions for students with autism, Mirenda (2003) summarizes findings from a meta-analysis conducted by Schlosser and Lee. Schlosser and Lee (2000) found that unaided AAC approaches such as manual signing were significantly more effective than aided approaches with regard to acquisition. However, no differences were found with regard to either generalization or maintenance. It should be noted that very few studies directly compared manual signing to aided techniques and fewer than 10% of the participants met the authors' inclusion criteria for autism. In a comparison of visual teaching strategies for children with autism, Tissot and Evans (2003) point out that the bulk of literature pertaining to sign language dates from the 1970s and 1980s whereas studies utilizing specific materials-based systems do not appear until much more recently. Goldstein (2002) points out in his review of empirical studies published between 1982 and 2002 that very few studies actually explore the use of sign alone, signing without the use of vocal cues. Despite the argument for sign only interventions due to overselective responding by individuals with autism, Goldstein (2002) justifies this lack of research, "...because the likelihood of children producing speech is diminished" (p. 385). However, in her review of sign alone and sign plus speech interventions for nonverbal children with autism, Creekmore (1982) concludes that, "The existing fonn communication (AAC) interventions for students with autism, Mirenda (2003) no differences were found with regard to either generalization or maintenance. should ofliterature " ... because 16 literature provides inconsistent support for both the use of sign alone and sign plus speech in language training programs for autistic children. One could argue in favor of either side with respect to the general language characteristics of this population" (p. 51). She goes on to state that total communication may actually confuse rather than help a child who is unable to process in more than one mode. Bonvillian et al. (1981) concur that data do not support the use of simultaneous communication over sign only techniques, and ".. .compelling arguments can be made that such a procedure might hinder some autistic children" (p. 133). It is necessary to analyze and synthesize a larger body of literature to help determine if total communication interventions result in better comprehension and production of both signs and speech as is argued by Goldstein in his review. Aided Augmentative and Alternative Communication (AAC) "Aided communication incorporates devices that are external to the individuals who use them (e.g., communication books and VOCAs) and involves the use of symbols such as photographs, line drawings, letters, and words" (Mirenda, 2003, p. 204). These alternatives to teaching a child to use his/her voice as the primary means of communication have been and continue to be developed with ongoing improvements in technology. Key adaptations include Electronic Speech (Voice) Output Devices and Symbol Boards. Electronic Speech (Voice) Output Devices employ synthetic digitized speech where ".. .voice is produced by a (rule-based) text-to-speech system that converts an input string of text characters to an output speech wave form" (Allen, 1992, p.741). Symbol Boards allow individuals to use pictures or symbols as a primary means of communicating or as a way to supplement or augment existing speech. " ... reVIew. hislher where" ... oftext 17 Voice Output Communication Aids (VOCAs) "VOCAs are portable electronic devices that produce synthetic or digitized speech output. A variety of graphic symbols can be used in conjunction with VOCAs to represent messages that are activated when an individual uses a finger, hand, or some other means to select a symbol from the VOCAs display" (Mirenda, 2003, p. 210). As technology increases, the capabilities of such speech generating devices also improve. Current research shows that individuals with autism can be taught to use such devices when the device is properly selected. According to Durand (1999), selection should be based on both an individual's current and future skills, such as the ability to point to one or more symbols, and the quality of the device's output. Mirenda (2003) finds that aided approaches must match environmental demands and listener capabilities. It is thought that the immediate voice output may serve as reinforcement to the individual using the device, and it is documented that individuals are more likely to be responded to by members in their community (Durand, 1999). In summary, Mirenda (2003) suggests that the argument in favor of aided techniques over manual sign may be legitimate for individuals with fine motor deficits. Word or Activity Boards or Schedules "One of the most common augmented input strategies involves the use of pictorial or written schedules to assist individuals to understand and follow predictable activity sequences in school and home settings (Quill, 1997; Wood Laskar, Siegal-Causey, Benkelam, & Bell, 1998)" (Mirenda, 2001, p. 143). Cafiero (2001) notes that augmented language input can be pictorial, text-based or a combination of the two. She notes that visually-based media are being used to both augment expressive language for individuals when the device is properly selected. According to Durand (1999), selection should be ofthe with autism and to provide comprehensible language input, the latter being more important in her opinion. This augmentation of language is often linear and does not represent typical language, but it is felt to capitalize on visual-spatial processing skills while minimizing auditory processing skills. These activity schedules are also used in much the same way as social stories. Pictures are arranged on environmentally specific language boards which serve as visual prompts to teach appropriate social communication. Schedules are also conducive to teaching people to make choices. For example, if individuals lack sufficient verbal skills to indicate preferences, they can select from an (initially small) array of pictorial or textual cues that represent different activities, add the selected activities to their schedules, and then engage in the chosen activities (MacDuff, Krantz, & McClannahan, 1993; McClannahan & Krantz, 1999). Picture Exchange Communication System (PECS) The Picture Exchange Communication System (PECS) is an augmentative or alternative communication program that combines the use of a symbol book and AB A-based methods to teach functional communication to children with limited speech. Developed at the Delaware Autistic Program by Lori Frost and Andrew Bondy, PECS teaches children to exchange a picture for something they want - an item or activity. This is taught through a hierarchy of phases (ASA, 2002; Tissot & Evans, 2003). As previously referenced, Creedon (1973) stresses the need for the most expedient form of communication training. Bondy and Frost (2001) highlight that PECS is rapidly acquired and notes, "Because tangible outcomes initially are more motivating to children with autism than are social outcomes, PECS begins by teaching requesting" (p.727). Creedon (1973) also acknowledges that a communication system must provide many opportunities 18 different 1 993; ABA­based PEeS 19 for consequences to be attached. By pairing the successful completion of a task (the picture exchange) with access to desired items, PECS provides such opportunities and may set the foundation for a self-initiated functional communication system. With the ultimate goal of spontaneous or unprompted communication, Bondy and Frost (2001) explain how the use of two trainers helps achieve spontaneity very early in the PECS program. "When teaching communicative initiation, two trainers are used: one who acts as the communicative partner (the listener) to interact socially with the child and one who provides physical prompting from behind..." (p.728). According to Bondy and Frost, this type of prompting reduces prompt dependency and is easily faded. Although "The system uses basic behavioral principles and techniques such as shaping, differential reinforcement, and transfer of stimulus control via delay to teach children functional communication using pictures (black-and-white or color drawings) as the communicative referent" (Charlop-Christy, Carpenter, Le, LeBlanc, & Kellet, 2002, pp.213-214), until recently, there have been very few published, empirical studies to test the efficacy of using PECS with children with autism. Informational reports do note improvements in spoken language in the form of both spontaneous and imitative speech along with collateral outcomes such as decreases in problem behaviors following PECS training (Bondy, & Frost, 1994; Krasny 2003; Schwartz, Garfmkle, & Bauer, 1998). Unfortunately, experimental research designs that could factor out maturation have not always been utilized. Most recently, several researchers (Adkins, & Axelrod, 2001; Anderson, 2002; Tincani, 2003) have begun comparing the use of sign langue or total communication with that of PECS on language acquisition and generalization. In each of these studies, the behind ... " this type of prompting reduces prompt dependency and is easily faded. Garfinkle, ofPECS 20 researchers are comparing the effects of topography-based communication with that of selection-based communication. Although a thorough analysis is beyond the scope of this particular study, "In topography-based systems, the topography of response varies between responses.... Sign language is a topography-based system because each sign has a different form (e.g., the sign for ball is different than the one for dog)..." (Bondy, Tincani, & Frost, 2004, p.258). In contrast, PECS is a selection-based system. "In selection-based systems, all responses are topographically similar and involve the selection of an appropriate stimulus from an array" (Bondy et al., 2004, p.258). Facilitated Communication (FC) Facilitated Communication (FC), an alternative communication system, involves a facilitator who, by supporting an individual's hand or arm, helps the person communicate through the use of a computer or typewriter (ASA, 2002). FC was developed in Australia by Rosemary Crossley and has been the topic of much controversy. FC is based on the idea that the person is unable to communicate because of a movement or neurological disorder, not because of a lack of communication skills. Douglas Biklen introduced FC to the United States and Canada in 1990, "Biklen maintains that individuals with autism suffer from a neurological disorder called dyspraxia, which interferes with the production of speech, and that facilitated communication allows autistic individuals to overcome this condition and communicate at a level that suggests that they are not intellectually impaired" (Montee, Miltenberger, & Wittrock, 1995, p. 189). Anecdotal claims profess that a trained facilitator is able to help an individual communicate what is otherwise trapped inside. Although, empirical research has not shown increases in communication skills, but rather facilitator responses .... [onn dog) ... " & aI., Fe 189). 21 interference (Kerrin, Murdock, Sharpton, & Jones, 1998; Mostert, 2001), some proponents of FC continue to refute the results. Environmental Modifications Treatment and Education of Autistic and Related Communication-Handicapped Children (TEACCH). Treatment and Education of Autistic and Related Communication- Handicapped Children (TEACCH) was developed at the School of Medicine at the University of North Carolina in the 1960s by Eric Schopler, Ph.D. TEACCH is a flexible approach to working with individuals with autism that is based on individual assessment and focuses on clearly organized, structured, modified environments and activities with an emphasis on visual learning modalities. TEACCH encompasses functional contexts for teaching concepts and uses structure and predictability to promote spontaneous communication (ASA, 2002). TEACCH, also known as the "parents-as-cotherapists" program, is a preventive program in which the environment and curriculum are structured to accommodate weaknesses of autism and capitalize on strengths (Miller, 2003). Although instructional materials are visually organized and verbal language is minimized, functional communication goals can be explicitly taught. Outcome studies demonstrate low rates of institutionalization, high rates of successful adaptation to the community and independent living (Miller, 2003). To date, few independent empirical studies have focused specifically on the acquisition of functional communication skills. Communication-Communication­Handicapped AS A, Relationship Based or Social Interventions Academic Inclusion Academic Inclusion, also known as PL 94-142, REI and IDEA, is an intervention based not only on the least restrictive learning environment but also on social learning theory. Inclusion assumes that children with autism will learn best in an environment surrounded by their peers; therefore, academic placement is inclusive, i.e. in regular education classrooms with normally developing children. Instruction may also entail a 1: 1 aide and curriculum modifications making it difficult to decipher which accommodations are benefiting which children and why. Imperative to its success, inclusion requires a team-approach to planning and implementation (ASA, 2002). Video Modeling Video Modeling is an intervention that incorporates many of the same behavioral principles as direct instruction but also relies on a child's ability to learn from peer models. Frequently thought of as an adjunct intervention, video modeling has shown positive outcomes for children diagnosed with other disorders such as attention deficit hyperactivity disorder (ADHD). Although the bulk of autism research focuses on direct instruction, video modeling has resulted in positive gains for some children with autism. Studies involving individuals with autism find that there is an ".. .overall lack of preference for self versus other video treatment for the group of participants as a whole" (Sherer et al., 2001, p. 152). Additionally, not all participants reach criterion even after months of treatment. 22 1 " ... aI., 23 Developmental Individual-Difference, Relationship-Based Model (DIR) Developmental, Individual-Difference, Relationship-Based Model (DIR) also known as Greenspan or Floor Time was developed by child psychiatrist Stanley Greenspan and targets emotional development following a developmental model that includes spontaneity along with semistructured play and motor-sensory play (ASA, 2002). This home-based approach to expanding purposeful communication in children with autism is often conducted concurrently with other interventions. "The DIR approach appears attractive to parents because of its focus on the atypical emotional reciprocity characteristic of autism" (Romanczyk, Arnstein, Soorya, & Gillis, 2003, p.377). Parents report improved communication between themselves and their children with autism, but specific dependent variables have not been put to rigorous experimental controls. Outcome studies are criticized for their use of "unvalidated' outcome measures and their use of a single clinician who was not blind to the hypothesis. Social Stories Social Stories or Social Scripts were developed by Carol Gray as a way to teach social skills to children with autism. Stories are written to address individual problem situations and are introduced to the child in advance of the situation to help clarify social expectations by reading and rereading the story or script. The stories are typically comprised of three types of sentences: perspective, descriptive and directive and often times incorporate pictures, photographs or music (ASA, 2002). To date, there is very little empirical support for many of these social or relationship-based interventions. Developmental, ofthree ofthese Dietary Interventions Many parents who have children with autism report decreases in autistic-like behaviors from their children after dietary changes. These changes can include the addition of vitamin and mineral supplements such as Vitamin B6 with Magnesium, Cod Liver Oil Supplements with Vitamins A and D, and Vitamin C, or the removal of gluten and/or casein from their children's diets. Although parent-report is critical in assessing children with autism, it is necessary to test such claims scientifically considering that "elimination diets" can place children at risk for malnutrition, and magadoses of many vitamins can result in toxicity, physical dependency and withdrawal symptoms. "If children are placed on GFCF (gluten-free casein-free) diets, they need to be monitored for iron status, calcium, and protein intake. Nutritional consultation is recommended, with careful monitoring of the child's growth and health by the practitioner" (Hansen & Ozonoff, 2003, p.l90). Pharmacotherapy Although many medications are used to treat comorbid conditions in children with autism (stimulant medications for attention problems; selective serotonin reuptake inhibitors (SSRIs) for compulsions, rituals, depression, anxiety, etc.; antipsychotic medications; neuroleptics, etc.), certain medications have been touted as "cures" for autism (Handen, 1993). Current literature explores the efficacy of secretin injections and chelation therapy for decreasing autistic symptomatology in children (Kidd, 2002). To date, secretin studies utilizing both treatment and control groups do not result in significant statistical differences between groups (Richman, 1999). Hansen and Ozonoff 24 p.190). 25 (2003) point out that there are ".. .pharmacological interventions being used for children with ASD for which there is little empirical evidence of efficacy" (p. 188). Study Rationale Although there is a plethora of research regarding various teaching techniques, interventions and the ever-so-promising "cure" for autism, this information is not readily available for interpretation by service providers, caregivers or educators. As is evidenced from the variety of interventions utilized for children and young adults with autism, it is difficult to summarize such a large and diverse body of literature. There is enough overlap between the different interventions that it is unclear which specific components hold the most promise for increasing functional communication skills in children with autism. Similarly, there are a variety of outcome measures that are used to assess specific areas of communication. Presently, it is not determinable from the research which specific language skills may be enhanced by a particular intervention. Finally, it is critical to be able to determine if subgroups of children with pervasive developmental disorders are differentially impacted by specific interventions. In an attempt to simplify a portion of the autism literature, this study will analyze and synthesize the Picture Exchange Communication System (PECS), sign only communication (SIGN), and total communication (TC) interventions for children and young adults with autism. Meta-Analysis Meta-analysis is one means by which the above concerns can be addressed. Meta-analysis is a perspective as much as a set of techniques that if used properly, can help consolidate the outcomes found in a large, diverse body of literature. In his Review are" ... PEeS), Review 26 of Developments in Meta-Analytic Method^ Bangert-Drowns (1986) expresses that, "Valuable information is needlessly scattered in individual studies" (p.398). One of the many benefits that meta-analysis provides over a qualitative literature review is that of objectivity. Cook and Leviton (1980) state, "Meta-analysis has a distinct advantage over qualitative reviews when a large number of studies exist" (p.460). Research integration should not be treated lightly and meta-analysis provides a rigorous means to accomplish such a goal; "It (meta-analysis) is rooted in the fundamental values of the scientific enterprise: replicability, quantification, causal and correlational analysis" (Bangert- Drowns, 1986, p.398). Although meta-analyses are not replete with criticism, this flexible set of tools continues to evolve, and as Glass (1978) notes, "In a field that lacks standard units of treatment and measurement, such as social psychology, all empiricism and reasoning is a problem at some level of coping with incommensurables" (p.359). It is presumed that findings from meta-analyses pertaining to treatment effectiveness have resulted in more accurate, detailed, and accessible bodies of scientific knowledge that can help influence public policy (Shadish, Chacon-Moscoso, & Sanchez-Meca, 2005). The National Research Council (2001) acknowledges, "Thus, while substantial evidence exists that treatments can reach short-term specific goals in many areas, gaps remain in addressing larger questions of the relationships between particular techniques, child characteristics, and outcomes" (p.217). Meta-Analysis of Single-Subject Research According to Allison and Gorman (1993), "One limitation of much past meta-analytic research is the exclusion of single case design" (p.621). This may have been due Analytic Method, infonnation should not be treated lightly and meta-analysis provides a rigorous means to accomplish such a goal; "It (meta-analysis) is rooted in the fundamental values ofthe scientific enterprise: replicability, quantification, causal and correlational analysis" (Bangert­Drowns, 1986, p.398). tenn Gonnan meta­analytic 27 to a lack of methodology to include such studies. Summary statistics such as means and standard deviations, which are necessary to calculate effect sizes, are often not published. Researchers are more likely to graph their data as a means of reporting single-subject results. Techniques involving visual inspection and drafting have been developed to convert graphs back to raw data, allowing for the calculation of the statistics necessary to compute an effect size. With the development of such techniques, researchers have recently focused on applying meta-analysis to single-case studies (Connelly, 2004, 2006; Maughan, 2004; Miller, 2006; Vegas, 2005). The goal of much of the research conducted in the social sciences is to identify individual changes over time. Since autism presents with such variability among individuals, it would be negligent to exclude such studies. In fact, Lord (2000) states, "Most of the highest quality research in the area of communication interventions in autism is based on single subject designs often replicated across a small number of subjects" (p. 395). Currently traditional literature reviews remain the norm for single-subject language interventions specific to autism (Bonvillian et al., 1981; Mirenda, 2001; Oxman et al., 1978). In addition to including single case design, having a well-defined purpose, identifying study inclusion standards, and specifying both independent and dependent variables at the onset, help to clarify the interpretable results of a meta­analysis. Statistical Analysis of Single-Subject Designs Allison and Gorman (1993) emphasize that, "One of the most difficult aspects of conducting a meta-analysis is determining effect sizes" (p.621). A variety of meta-analytic methods have been utilized with single-subject research. Some of the more compute an effect size. With the development of such techniques, researchers have recently focused on applying meta-analysis to single-case studies (Connelly, 2004, 2006; Maughan, 2004; Miller, 2006; Vegas, 2005). single­subject aI., aI., meta­analytic 28 common techniques include (a) percentage of nonoverlapping data (PND) (Scruggs, Mastropieri, & Casto, 1987); (b) percent zero data (PZD) (Scotti, Evans, Meyer, & Walker, 1991); (c) ITSACORR (Crosbie, 1993); (d) Hierarchical Linear Model (Bryk & Raudenbush, 1992). Percentage of Nonoverlapping Data (PND) One method for conducting a meta-analysis of single-case data is to compute the percentage of nonoverlapping data between conditions. The PND is calculated by dividing the number of data points in the treatment phase which exceed the highest data point in the previous baseline phase by the total number of data points in the treatment phase and multiplying the result by 100. This nonparametric method, developed by Scruggs et al. (1987), has received criticism for "...erroneously represent(ing) effects in at least three situations: (a) when outliers are present in the baseline phase, (b) when the treatment has a detrimental effect, and (c) when trend is present in the data. In addition, PND is, at best, sensitive to changes in level but ignores changes in slope" (Allison, & Gorman, 1993,p.622). Percentage of Zero Data Percentage of zero data (PZD), developed by Scotti et al. (1991), is a variation of the PND. The PZD is calculated by dividing the number of treatment phase data points (1993) do not recommend the PZD because it is only usable with ratio scale data, and it is only useful when the goal is behavior elimination. nonoverlap ping Nonoverlap ping " ... erroneously In PND is, at best, sensitive to changes in level but ignores changes in slope" (Allison, & Gorman, 1993, p.622). oftreatment equal to zero by the total number of treatment phase data points. Allison and Gorman 29 ITSACORR Interrupted Time-Series Analysis for Autocorrelated Data (ITSACORR) is a computer program developed by Crosbie (1993) that removes the influence autocorrelation has on single-subject data sets, thereby controlling for Type I errors. ITSACORR produces an overall F statistic which indicates the probability of change between the baseline and treatment phases. If significant, ITSACORR provides separate r-tests to measure change in intercept and change in slope. Several meta-analysts have hypothesized that ITSACORR results in effect sizes that are too conservative, understating treatment effectiveness (Maughan, 2004; Miller, 2006; Vegas, 2005). Hierarchical Linear Model (HLM) Often referred to as multilevel modeling, Hierarchical Linear Model (HLM), developed by Bryk and Raudenbush (1992), distinctively accommodates data structures from the social sciences that are naturally nested or hierarchical. A commonly cited example of this phenomenon is education; students are grouped in classes which are grouped in schools which are further grouped in districts and so on. HLM is based on the assumptions of linearity and normality with an adaptation to ensure independence of observations, a basic for classical statistical techniques. In general, individuals within the same group are more similar than individuals in different groups. By using group and individual variance component models, individuals in different groups can be treated as independent while individuals within the same groups will inherently share values on some variables (Leeuw, 2002). This adaptation makes it natural to apply HLM to single-subject meta-analytic data where observations are nested within participants who are nested within studies. HLM provides for models that take variation at the subject and subj ect t-tests nonnality single­subject 30 study level into account, facilitating a meta-analysis which is both statistically correct and not wasteful of potentially valuable information. In the quantitative synthesis of findings from many studies, investigators may explore how differences between studies in treatment type, implementation, research methods, and participant characteristics effect estimates within studies. HLM provides the statistical framework for these research activities (Berkey, Hoaglin, Mosteller, & Colditz, 1995; Morris & Normand, 1992; Raudenbush & Bryk, 1985). Prior to testing for the effects of an independent variable, a preliminary random effects analysis of variance (ANOVA) is conducted with no predictor variables to test for variance among studies in mean effect sizes. The level 1 equation for the random effects ANOVA is: Yij = Pj eij where Yy = the individual effect size for participant i, study j ; pj = the mean effect size for study j ; and ey = a residual. In effect, this level 1 model represents a regression equation for each individual. It is considered a null model because it contains no predictor variables. The level-2 null model follows: Pj = 7o + r0 j where yo is the grand mean effect size and rqj represents the difference between the effect size for study j and the grand mean effect size (yo). Predictor variables such as participant characteristics may be added to the level-1 model, and study characteristics such as types infonnation. Nonnand, Raudenbush & Bryk, 1985). ANOVAis: Y ij = j; ~j = j; eij Yo rOj Yo). 31 of treatments, etc., may be added to the level-2 model. The level-1 variables are used to test if the effects of treatment are stronger for some participants than others. Prior to including level-2 moderator variables, a % is calculated to determine if the variance in obtained effect size scores across studies is due to chance. Only if the %2 is found to be significant, are moderator variables added to the level-2 equations to explain the variance among and studies. Examining and Reducing Bias In order to determine the number of null findings (e.g., ES 0) that would be necessary to reduce a significant effect size to a nonsignificant or "small" effect size, a fail-safe TVcan be computed in accordance with the following formula described by Hunter and Schmidt (1990): x = k ( d k / d c - l ) where k = the number of effect sizes included in the meta-analysis; dk = the effect size computed in the meta-analysis; and dc = the critical value for the effect size. Purpose The purpose of this study is threefold: first, to systematically search and synthesize the published research pertaining to functional communication interventions for individuals diagnosed with autism. In order to identify successful interventions, the studies will need to assess outcomes in at least one of the core areas of functional communication such as global, receptive or expressive language skills. These skill areas can be measured in a variety of ways. Changes in standard language scores, although "l ifthe X2 = N can = = de = functional 32 rarely reported, provide for normative comparisons. Changes in the number and accuracy of correctly identified words, symbols, and signs, that are used to communicate, can be a measure of receptive language fluency. Konstantareas (1977) defined receptive sign language as, "Signs the child, through compliance or other behavior, is able to understand. To test understanding verbalization is omitted" (p. 18). Similarly, changes in the number of words, symbols, and signs that are utilized by an individual to communicate can be a measure of expressive language fluency. For communication to be considered spontaneous, a child must produce a sign, vocalization or picture exchange on his or her own, in context and without prompting. Konstantareas (1977) exemplifies, "When the child signs 'help' when he has difficulty tying his shoes" (p. 18). The second purpose of this study is to better identify specific interventions for children and young adults with autism which result in increased frequency and/or accuracy of words, pictures, symbols, or signs used to communicate. Finally, this study hopes to determine if subgroups of children and young adults with pervasive developmental disorders are differentially impacted by specific interventions. Nonlanguage measures will not be statistically analyzed for purposes of this meta-analysis. A statistical analysis will be performed on the available data using the computer program, Hierarchical Linear Modeling (HLM). Effect sizes (ES) will be calculated for PECS, Sign Only Communication and Total Communication interventions described in the research. This common metric will allow for comparisons to be made between treatments while taking individual differences into account. By consolidating and analyzing the available research, effective interventions may be identified which can help lead to better practices. Parents, teachers, clinicians and care providers, in addition to nonnative ofreceptive communicate can be a measure of expressive language fluency. For communication to be considered spontaneous, a child must produce a sign, vocalization or picture exchange on his or her own, in context and without prompting. Konstantareas (1977) exemplifies, purpose of this study is to better identify specific interventions for children and young detennine perfonned 33 individuals with autism, stand to benefit from a systematic evaluation of the treatment modalities available for increasing functional communication skills in children with autism. Research Questions 1. Is the global effect size (ES) of interventions used to increase functional communication skills in children and young adults with autism different from zero? 2. Are the mean or composite effect sizes of interventions (total communication, sign only communication, and PECS) used to increase functional communication skills in children and young adults with autism different from zero? 3. Do the interventions used to increase functional communication skills in children and young adults with autism affect language type (expressive, receptive, and spontaneous)? 4. Does treatment effectiveness vary as a function of language type (expressive, receptive, and spontaneous) and treatment type (total communication, sign only communication, and PECS)? 5. Do the following participant characteristics moderate the effects of interventions used to increase functional communication skills in children and young adults with autism? a. Adaptive score b. Age in months c. Diagnosis d. Gender e. IQ score f. MR category g- Receptive language score h. Verbal classification 6. Are there interaction effects between coded participant variables (adaptive score, age, diagnosis, gender, IQ, MR category, receptive language score, and verbal classification) and each treatment type (total communication, sign only communication, and PECS) used to increase functional communication skills in children and young adults with autism? 7. Do the following study characteristics moderate the effects of interventions used to increase functional communication skills in children and young adults with autism? a. Data base b. Journal functional spontaneous) ? PEeS)? g. 34 c. Year d. Research Design e. Number of observations f. FBA (functional behavior assessment) Generalization component h. Measurement type (frequency or proportion) i. Reliability J. Implementer k. Setting Intensity in minutes per day m. Duration in weeks Does treatment effectiveness vary as a function of coded study variable (FBA, generalization, measurement type, reliability, implementer, and setting) and each treatment type (total communication, sign only communication, and PECS) used to increase functional communication skills in children and young adults with autism? g. 1. J. 1. 8. PEeS) CHAPTER II METHOD Design The design of this study was nonexperimental and utilized meta-analytic techniques to analyze and synthesize a large and diverse body of literature comprised of experimental and quasi-experimental studies published between 1965-2004. Literature Search Procedure Potential studies for inclusion in the meta-analysis were retrieved through computerized searches in Academic Search Elite, ERIC, Psychlnfo, PubMed, Medline, Dissertation Abstracts International, and ProQuest databases along with reference lists of relevant articles, review articles, chapters and books. EBSCO database was used as backup database to ensure a comprehensive search. The following descriptors were used in the computerized searches: autism, autistic, autism spectrum disorder, ASD, autistic-like, pervasive developmental disorder, PDD, Asperger's, Asperger's syndrome, childhood schizophrenia, Kanner's syndrome, language, speech, communication, functional communication, research, empirical, treatment, and intervention. The search also included the following authors, who were assumed to have published numerous studies on the topic of functional communication and autism: Carr, Koegel, Konstantareas, LoVaas and Prizant. Additional individualized hand searches were The design ofthis study was nonexperimental and utilized meta-analytic techniques to analyze and synthesize a large and diverse body of literature comprised of experimental and quasi-experimental studies published between 1965-2004. PsychInfo, Pro Quest autistic­like, 36 and Journal two journals that had the greatest frequency of articles based on the computer search. Hand searches were also conducted for all additional articles referenced in research articles located through the database and journal searches. The literature search was limited to studies conducted between 1965 and 2004. In order to eliminate studies that clearly did not meet inclusion criteria, all abstracts were examined prior to study selection. Criteria for Study Inclusion Studies were considered for inclusion in the meta-analysis if the abstract described a quasi-experimental or experimental design with at least one dependent measure related to language or functional communication such as number of pictures or objects correctly identified, number of words spoken or gestures signed, or change in standard language scores from baseline. To be included in the meta-analysis, a study met the following criteria: 1. Study must have been written in English. 2. Participants in the study must have fit the DSM-IV criteria for any of the pervasive developmental disorders, excluding Rett syndrome, Symbiotic Psychosis and Disintegrative Disorder. Participants with dual diagnoses were included as long as one of their diagnoses included Autism, Asperger's, or Pervasive Developmental Disorder. 3. Participants in the study must have ranged in age from 18 months to 22 years, 11 months. conducted in the Journal of Applied Behavior Analysis and the Journal of Autism and Developmental Disorders/Journal of Autism and Childhood Schizophrenia, the two ofthe 37 4. The outcomes of the study must have examined at least one of the core areas of communication such as Composite, Receptive or Expressive Language skills. 5. The study must have been a journal article, book or chapter, professional paper, thesis, or dissertation either published or unpublished between 1965 and 2004. 6. The study must have used a single-subject research design: an experimental design that documents changes in an individual subject's behavior, allowing for the demonstration of functional relationships between the interventions and effects on behavior. 7. The study must have provided adequate quantitative information to permit calculation of an effect size statistic. 8. Each single-subject design study must have reported or provided graphs displaying raw baseline and treatment data, which are necessary for the calculation of an effect size. Each participant was included on only one occasion. If a study targeted multiple behaviors or was conducted in multiple settings, only one behavior and setting were chosen per study. Raw data from one dependent variable per study was coded for analysis. If more than one dependent variable was included in a study, this author chose the most complete data representing the highest level of language functioning, i.e., spontaneous over expressive over receptive. 9. Each single-subject study must have used one of the following research designs (as defined by Kazdin, 1982; Kazdin, Bass, Ayers, & Rodgers, 1990; Tawney & Gast, 1984): (1) A1B1A2 designs', which collect baseline data for a participant on a target behavior until a stable baseline is obtained. Next, a treatment or intervention ofthe ajournal AJBJA2 designs: 38 is implemented and data are collected until stability is once again obtained. Finally, the intervention is withdrawn and the baseline phase is restored. (2) A1B1A2B2 designs', which add an additional treatment phase to the A1B1A2 design. (3) Multiple baseline designs: a. Multiple baseline across settings', which track one behavior across a minimum of three independent settings until stable baselines are established in each setting. Next a treatment or intervention is introduced in one setting, while baseline conditions are continued for the others. Once the criterion level is reached in the first setting, the procedure is repeated in each remaining setting. participants', behavior under the same conditions for a minimum of three participants until a stable baseline is obtained. Next an intervention is introduced to one participant while baseline conditions are continued for the others. When the first participant's behavior reaches the criterion level, the procedure is repeated until all participants receive the intervention. behaviors', three independent behaviors under the same condition until stable baselines are obtained. An intervention is then applied to the first behavior while data continue to be gathered for each behavior. Once the first behavior reaches the criterion level and the other restored. AlB lA2B 2 designs: AJBJA2 settings: b. Multiple baseline across participants: which monitor the same c. Multiple baseline across behaviors: which track a minimum of 39 behaviors have remained stable, the intervention is initiated for the second behavior. At this point both the first and second behaviors are receiving the intervention, and data continue to be collected for each behavior. The process continues until the intervention has been applied to all behaviors. (4) Multiple probe designs: Although similar to multiple baseline designs, the amount of data collected at baseline and sometimes during treatment differs. Rather than collecting data continuously, assessment probes are conducted only occasionally. (5) Multitreatment designs (e.g., A1B1A2C1A3D1): condition until data on the target behavior have stabilized. Following the baseline, the first treatment is implemented until the target behavior again becomes stable. Once this occurs, the baseline phase is reinstated until data are stable. Next a second intervention is implemented. Once the target behavior stabilizes in this second intervention phase, a third baseline or a return to the first intervention phase is initiated. If a third treatment is introduced, it may be followed by a return to baseline, the first intervention or the second intervention. In order to be included in this meta-analysis, multitreatment designs must allow for a direct comparison of a treatment with a baseline phase. Changing criterion, alternating treatment and simultaneous treatment design studies were examined on a study-by-study basis for inclusion in this meta-analysis. Although difficult to transform several baseline and treatment phases into one grand AjBjA2CjA3DJ): which monitor a baseline 40 baseline and treatment phase, in their quantitative synthesis of single-subject research, Scruggs et al. (1988) were able to analyze alternating treatment design studies by evaluating relative acquisition slopes for each intervention included. For this meta­analysis, only one treatment intervention was measured for each study. Treatments were chosen based on completeness of data provided in the article as well as need for a particular intervention. For example, if a study provided adequate data for total communication and PECS, PECS was chosen as the treatment type due to the limited number of available PECS studies. The names and/or identifying information for each participant were tracked by author to avoid including participants on more than one occasion, ensuring that sample size was not inflated. It is assumed that applying the above selection criteria to the functional communication literature as it pertains to individuals with autism would result in approximately 40 studies being included in this meta-analysis with each individual being included on only one occasion. Method for Analyzing Research Articles Each study included in the meta-analysis was coded in Microsoft Excel® on the basis of specific characteristics of the article (see Appendix A for the Coding Sheet). The following study variables were coded: sources of study (both data base and journal, dissertation, etc.); year published (1965-2004); research design; treatment setting (school, home, clinic, residential/institution/hospital, or other); treatment type (total communication, sign only communication, and PECS); treatment implementer (teacher, parent, researcher/university student, therapist/clinician/MD, or staff); treatment intensity (minutes/day); treatment duration (weeks); and whether follow-up or maintenance, residentiallinstitution/therapisticlinicianIMD, 41 generalization, reliability, and FBA were conducted. Participant variables that were coded include chronological age (expressed in months); diagnosis/diagnoses; gender; level of cognitive functioning (global, verbal, nonverbal, and categorical); level of adaptive behavior functioning; level of language functioning (expressive and receptive); and verbal classification (nonverbal, verbal, and echolalic). Two coded variables were eliminated due to minimal reporting and/or lack of heterogeneity among participants: previous language intervention and primary form of communication. A second coding sheet (Appendix B) was developed to record individual data points from the graphs in each study which were then coded into Excel. Reliability of Data Collection independent rater, a professor in the college of Health and Human Services at a university in the midwest, used the same coding procedures as I to code 11 (25% of the obtained studies) randomly selected studies. This same independent rater provided a reliability check on 25 of the graphs of individual data points. In order to calculate the percentage of interrater agreement regarding the individual study and participant characteristics, interrater agreement was coded separately for each subject. The agreement rate (AR), or percentage of agreement, was calculated by taking the number of agreements and dividing them by the total number of data points (agreements plus disagreements). In light of the difficulty of reading graphed data in many of the studies, raters were considered in agreement on a given data point if they coded it within +/- 1 metric point of each other. ExceL An metric point of each other. 42 Computation of Effect Sizes Combination of Baseline and Treatment Phases In order to calculate one ES for each participant, multiple baseline phases and multiple treatment phases were consolidated into one baseline phase and one treatment phase per study, a form similar to an A1B1 design. Computation of Effect Sizes With Hierarchical Linear Model [HLM] An effect size was obtained for each participant in the following manner. A text file was created that contained the observed individual data points for baseline and treatment phases of one dependent variable per study. A custom Fortran program was written by Dr. John Kircher to read the text file and compute the mean and standard deviation of all baseline points and the mean and standard deviation of all points in the treatment phase. The effect size was the difference between the treatment mean and the baseline mean divided by the larger of the two standard deviations. Using the larger of the baseline and treatment standard deviations was more appropriate than using the pooled standard deviation due to the nature of the construct being measured and the severe communication deficits of the participants included in the treatment interventions. The majority of individuals identified as nonverbal had baseline language measures equal to zero. This method for calculating a standardized effect size provided the most conservative (smallest) effect size for each participant. An adjustment was made for the participants of three AB studies due to the limited number of data points reported. The adjusted effect size was the difference between the individual's treatment mean and baseline mean divided by the study or group standard deviation. Each study's standard AlBI study'S 43 deviation was calculated using the treatment data points reported for all study participants. Again, this generated the most conservative effect size for each individual. The effect sizes for all study participants were entered into a level-1 data file for the HLM analysis. The level-1 data file also included participant characteristics, such as age and IQ score. A level-2 data file also was created that identified the participants in each study and included study characteristics, such as treatment setting and intensity. To maximize the number of cases for each statistical test, different level-1 and level-2 data files were created for each independent variable. For example, when testing for effects of age on treatment effectiveness, all participants for whom the variable age was available were used to create level-1 and level-2 data files. Participants for whom age was not available were excluded from the analysis. This allowed for a two-level hierarchical analysis of data specific to the moderating effects of participant age on treatment effectiveness. Different subsets of cases were similarly selected for analyses of all participant and study characteristics. Prior to testing for the effects of an independent variable, a preliminary random effects analysis of variance (ANOVA) was conducted with no predictor variables to test for variance among studies in mean effect sizes. The level-1 equation for the random effects ANOVA was: where Y§ the individual effect size for participant i, study j ; pj the mean effect size for study j ; and = a residual. In effect, this level-1 model represents a regression equation l l l l l ANOV A ~j = j; Pj = j; eij = l 44 for each individual. is considered a null model because it contains no predictor variables. The level-2 null model was as follows: Pj = Yo + r0j where yo was the grand mean effect size and roj represented the difference between the effect size for study j and the grand mean effect size (yo). Predictor variables such as participant characteristics may be added to the level-1 model, and study characteristics such as types of treatments, etc., may be added to the level-2 model. The level-1 variables are used to test if the effects of treatment are stronger for some participants than others. Prior to including level-2 moderator variables, a %2 was calculated to determine if the variance in obtained effect size scores across studies was due to heterogeneity among studies beyond what would occur by chance alone. Only if the %2 was found to be significant at the .05 level, were moderator variables added to the level-2 equation to explain the variance among the studies. In addition to the %2 statistic, the intraclass correlation coefficient (ICC) was calculated to determine the proportion of variance in the dependent variable that was accounted for by study differences. All HLM final estimations of variance components are tabled in Appendix C. The ICC was calculated from the following formula described by Luke (2004): 2 2 2 p = o u 0 / ( a uO + CJ r ) 2 2 where a u 0 and o r are estimates of the level-2 and level-1 variances, respectively and were obtained from fitting a null-model using HLM. It Yo rOj effect size for study and the grand mean effect size (yo). Predictor variables such as participant characteristics may be added to the level-l model, and study characteristics l X2 X2 significant at the .05 level, were moderator variables added to the level-2 equation to explain the variance among the studies. In addition to the X2 statistic, the intraclass correlation coefficient (ICC) was calculated to determine the proportion of variance in the dependent variable that was accounted for by study differences. All HLM final ICC where cr2 uoand cr2 r are estimates of the level-2 and level-l variances, respectively and were obtained from fitting a null-model using HLM. 45 R2 values were calculated from the following formulas provided by Raudenbush and Bryk (2002): 2 2 2 R 1 ~ (g rNull I O" r C o m p a r i s o n 1 O" rNull 7 7 where a r N un represents the total within-study variance and o r C o m p a r i s o n represents any change in the within-study variance that can be accounted for by predictor or moderator variables. Similarly, the proportion variance explained at level-2 can be calculated from the following formula: 2 2 2 R 2 = ( g L u N u l l " O" uComparison ) O" uNull 2 • 2 where o U N u i i represents the total between-study variance and a uComparison represents any change in the between-study variance that can be accounted for by predictor variables. Computation of a Fail-Safe TV In order to determine the number of null findings (e.g., ES = 0) that would be necessary to reduce a significant effect size to a nonsignificant or "small" effect size, a fail-safe TV was computed in accordance with the following formula described by Hunter and Schmidt (1990): x = k ( d k / d c - l ) where k = the number of effect sizes included in the meta-analysis; dt = the effect size computed in the meta-analysis; and dc = the critical value for the effect size. In addition 2 (",2 2 \ R 1 = lJLrNull - () rComparison J. -----r 0' rNull where ()2rNull represents the total within-study variance and 0'2 rComparison represents any change in the within-study variance that can be accounted for by predictor or moderator variables. Similarly, the proportion variance explained at level-2 can be calculated from the following formula: (",R 2 = !S!...uNull - 0' uComparison} -----z- () uNull where ()2 uNull represents the total between-study variance and 0'2 uComparison represents any change in the between-study variance that can be accounted for by predictor variables. Computation of a Fail-Safe N In order to determine the number of null findings (e.g., ES = 0) that would be necessary to reduce a significant effect size to a nonsignificant or "small" effect size, a fail-safe N was computed in accordance with the following formula described by Hunter and Schmidt (1990): = k (dk / <L: - 1) = dk = de = 46 to calculating a fail-safe N for the global ES, a fail-safe N was also calculated for each treatment type. CHAPTER III RESULTS The purpose of this study was to conduct a meta-analysis of the empirical literature in order to identify which interventions are most effective at increasing functional communication skills in children and young adults with autism. This study also investigated specific participant-level moderator variables (adaptive score, age, diagnosis, gender, intensity, IQ, MR category, receptive language score, nonverbal/verbal classification, and duration) along with study or treatment-level moderator variables (data base, journal, year, design, number of observations, FBA, generalization, measurement type, reliability, implementer, setting, and intensity) that might influence the effectiveness of the different intervention types. Characteristics of the Study A total of 12,381 studies were identified during the initial computerized literature search. All abstracts were read, and studies that did not address language or functional communication interventions, did not meet the inclusion criteria, or were not empirical were excluded. This initial screening yielded 1,585 potential studies that were entered into a database. To scale down the scope of this meta-analysis, specific functional communication interventions were chosen: sign only communication, total communication, and PECS. This reduction resulted in the need to collect and examine functional 48 Reasons for Excluding Studies From Meta-Analysis Reason for Exclusion Number of Studies Treatment type was not specific to TC, SIGN, or PECS 59 Not a treatment/experimental study (e.g., qualitative, descriptive case study, treatment review) 57 Single subject data not provided (e.g., group studies) 16 Different dependent variable (e.g., articulation, on-task behavior) 13 Not available from U.S. library system 12 Duplicate study (e.g., dissertation, same participant data reported in different articles) 10 Inadequate data to calculate an effect size (minimal or no raw data provided) CO Participants did not fit inclusionary criteria (e.g., incorrect diagnosis or age greater than 22 years) 6 224 studies. After examining each article, 43 or 19 % of the studies identified for further inspection met the inclusion criteria. Each study included in the meta-analysis is marked by an asterisk in the reference pages of this document. Table 1 provides a rationale for excluding the remainder of the studies from the meta-analysis. A summary of the characteristics of the included studies is provided in Table 2. For the 43 single-subject design studies included in the present meta-analysis, the total number of participants was 138. Characteristics of participants in the included Table 1 on­task 8 Table 2 Characteristics of Included Studies CHARACTERISTIC Total Number of Studies Total Number Participants Treatment Type SIGN TC PECS Dependent Variable (Language DV) Expressive Receptive Spontaneous Design Multiple Baseline AB Multiple Probe Other (multi-, alternating, and simultaneous treatment, ABCD, reversal, and changing criterion) Dichotomous Design MLB Other Data Base Psychlnfo ERIC PubMed/Medline ProQuest/Dissertation Abstracts Academic Search Elite Hand Search Journal JABA Other Journals J Autism & Dev Dis Professional Paper Book/Chapter Dissertation/Thesis Dichotomous Journal JABA Other Year (1965-2004) n k %k 43 138 43 16 8 18.60% 98 25 58.14% 24 10 23.26% 71 19 44.19% 3 3 6.98% 64 21 48.84% 46 19 44.19% 30 2 4.65% 45 12 27.91% 17 10 23.26% 46 19 44.19% 92 24 55.81% 39 4 9.30% 10 4 9.30% 48 16 37.21% 16 5 11.63% 4 2 4.65% 21 12 27.91% 20 7 16.28% 26 16 37.21% 28 9 20.93% 10 3 6.98% 32 2 4.65% 22 6 13.95% 20 7 16.28% 118 36 83.72% 49 n %43 multi·, DataBase PsychInfo PubMedlMedline ProQuestlDissertation Dichotomous Journal 1965·Table 2 Continued CHARACTERISTIC n k %k Functional Behavior Assessment W/O 127 39 90.70% With 11 4 9.30% Dichotomous Generalization W/O 74 15 34.88% With 64 28 65.12% Maintenance W/O 92 31 72.09% With 46 12 27.91% Measurement Type Frequency 106 29 67.44% Proportion 32 14 32.56% Reliability W/O 67 12 27.91% With 71 31 72.09% Implementer 137 41 95.35% Teacher 24 6 13.95% Parent 4 1 2.33% Researcher/University Student 67 26 60.47% Therapist/Clinician/MD 15 5 11.63% Staff 27 3 6.98% Dichotomous Implementer Teacher 24 6 13.95% Other 113 35 81.40% Setting 122 40 93.02% School 39 14 32.56% Home 4 1 2.33% Clinic 17 CO 18.60% Residential/Institutional/Hospital 45 11 25.58% Other 3 1 2.33% >1 Setting 14 5 11.63% Dichotomous Setting School 39 14 32.56% Other 83 26 60.47% Intensity in Min/Day 74 29 67.44% 50 n k %k W/O W/O W/O W/O ResearcherlUniversity TherapistiClinicianlMD 8 ResidentiallInstitutionallHospital MinlDay 51 studies are provided in Table 3. Summary statistics for all noncategorical moderator variables are provided in Table 4. Reliability Table 5 details the percentage agreement between the researcher and an independent rater on the coding of predictor or moderator variables and data points. Examination of Research Questions Research Question # 1: Is the global effect size of interventions used to increase functional communication skills in children and young adults with autism different from zero? Prior to including level 2 moderator variables, a %2 was calculated to determine if significant variance exists among studies. All analyses were run using an alpha level of 0.05 (p < 0.05). The results from the null model indicated significant heterogeneity, X2(42, N = 138) = 339.265,/? < 0.01). A global effect size was calculated using HLM 6.0 to determine if a significant treatment effect exists across all participants included in the meta-analysis. The fully unconstrained null model yielded an average effect size across all participants of 2.036 (SE = .304), t(42) = 6.689,/? < 0.01, with individual effect sizes ranging from -0.74 to + 11.81. According to standards provided by Cohen (1988), this represents a large treatment effect. In order to determine the proportion of variance (see Appendix C for variance components) in the dependent variable that was accounted for by groups (i.e., level-2 units or studies), the intraclass correlation coefficient (ICC) was calculated (a2 uo / ( o 2 ^ + different X2 X2(42, N= 339.265,p to determine if a significant treatment effect exists across all participants included in the meta-analysis. The fully unconstrained null model yielded an average effect size across of2.036 = = 6.689,p effect. ((i uO / (if-uO Table 3 Characteristics of Participants Included n k %k Total Number of Studies 43 138 43 100.00% 51 12 27.91% 122 41 95.35% 138 43 100.00% Autism 121 PDD 4 1 Autistic-Like 5 Multiple 7 138 43 100.00% Autism 121 Other 17 116 42 97.67% Female 19 Male 97 52 13 30.23% MR Category 63 20 46.51% 1 12 20 Severe 20-35 24 6 30 >35 33 70 13 30.23% 53 >35 17 Classification 123 41 95.35% 31 Nonverbal 82 Echolalic 10 Verbal 41 Nonverbal 82 Duration in Weeks 137 29 67.44% 52 CHARACTERISTIC k Total Number Participants Adaptive Score Age in Months Diagnosis l38 Childhood Schizophrenia Dichotomous Diagnosis Gender IQ Score MRCategory Normal IQ >70 Mild 50-70 Moderate 35-50 Profound <20 Dichotomous MR Category <35 ~35 Receptive Language Score Dichotomous Receptive <35 ~35 Verbal Classification Verbal Dichotomous Verbal Class 53 Summary Statistics for Moderator Variables Variable n Mean SD Minimum Maximum Adaptive Score 51 29.16 17.64 .10 74.80 122 87.69 42.20 23 192 Duration in Weeks 137 43.29 65.50 2.80 394.30 Intensity Min/Day 74 73.65 82.35 10 360 IQ 52 38.05 17.40 0 101 Receptive Language 70 26.40 19.86 0 138.5 138 26.42 34.97 2 168 Table 5 Reliability Agreement Rate Study Characteristics Agreement Rate 97% Treatment Type 100% 100% Language DV 100% Gender 100% Measurement Type 100% IQ 100% Data Base 98% MR Category 100% Year 100% Classification 90% Journal 100% 100% Design 82% 100% Treatment Setting 9 1% Duration in Weeks 81% Treatment Implementer 82% 100% Intensity in Minutes per 100% Day 97% Maintenance Component 100% Data Points 98% Generalization Component 91% Interrater Reliability 100% Table 4 Age in Months in IQ Number of Observations Participant Characteristics Diagnosis Age in Months IQ Verbal Classification Adaptive Score Receptive Language Score 91% Functional Behavior Assessment (FBA) Phase 54 a 2 r ) from the null model. The ICC [3.21054 / (1.32601 3.21054) = .707705] denotes that studies accounted for just over 70% of the variance among participants suggesting that a multilevel model incorporating level-2 studies and study characteristics may be necessary to explain the variance among effect sizes (Luke, 2004; Raudenbush & Bryk, 2002). Research Question #2: Are the mean or composite effect sizes for each intervention (total communication, sign only communication, and PECS) used to increase functional communication skills in children and young adults with autism different from zero? When treatment type was added to the null model, the resulting chi-square was significantly greater than zero, %2(40, N = 138) = 327.469,/? < .01), indicating significant heterogeneity among treatment types. In order to determine the expected mean effect size for each treatment type, three HLM 6.0 models were run. Table 6 displays the HLM 6.0 statistics by treatment type. The reported coefficient may be interpreted as a composite or mean effect size. Pair-wise comparisons indicated that there were no significant differences among the treatments. Table 7 depicts the HLM statistics for each comparison. Table 6 Composite Effect Size by Treatment Type Fixed Effects Coefficient SE T ratio Approximate p value TC 2.343 0.395 5.929 40 0.000 SIGN 2.051 0.708 2.894 40 0.007 PECS 1.242 0.630 1.970 40 0.055 cr2 r ) 3.21054/ + functional X2(40, N = 327.469,p companson. Effects Tratio df 55 Table 7 In order to further investigate the effectiveness of PECS as a functional communication intervention, measurement type was added to the level-2 model as a means of controlling for the difference in reporting outcome data among studies. More specifically, outcome data were reported either as frequencies or as proportions. Table 8 depicts counts for each treatment by measurement type. Figure 1 illustrates mean effect sizes for each treatment and measurement type. There was no significant main effect for measurement type (see Research Question #7h). Using Cohen's 1988 standards, each Table 8 Frequencies for Treatment by Measurement Type Measurement Type Treatment n k %k Frequency TC 18 72.00% SIGN 6 4 50.00% PECS 18 7 70.00% Proportion TC 16 7 28.00% SIGN 10 4 50.00% PECS 6 3 30.00% Pair-wise Comparisons for Each Treatment Type Fixed Effects Difference SE T ratio Approximate Rvalue Coefficient df D i f f e r e n c e from TC SIGN -0.292 0.811 -0.360 40 0.721 PECS -1.100 0.744 -1.479 40 0.147 Difference from SIGN PECS -0.808 0.948 -0.852 40 0.399 Tratio p value Coefficient df Difference from TC SIGN -0.292 0.811 -0.360 40 0.721 PECS -1.100 0.744 -1.479 40 0.147 Difference from SIGN functional effect 82 k 56 Mean Effect Size by Treatment and Measurement Type & Frequencies • Proportions • Composite SIGN PECS Figure 1. Mean Effect Size by Treatment and Measurement Type individual treatment type resulted in a large treatment effect. Table 9 depicts the HLM statistics when measurement type is controlled and total communication serves as the comparative (M 2.732, SE 0.431), t(37) 6.340,/? < 0.01. R2 2 was calculated from the following formula, provided by Raudenbush and Bryk (2002), to determine the proportion reduction in variance or "variance explained" by adding treatment type to the null model: Table 9 Pair-wise Comparisons With Measurement Type Controlled Fixed Effects Difference Coefficient SE T ratio Approximate /? value C o m p a r e d to TC SIGN -0.687 0.813 -0.845 37 0.404 PECS -1.577 0.794 -1.986 37 0.054 4 3.5 3 2.5 ~ 2 1.5 0.5 o Efiect TC 1m o = = = 6.340,p R22 from Effects Tratio p df Compared t 0 TC 57 .uNull ~ Q^uCotnparison 1 O" uNull Treatment type accounted for 1.36% of the between-study variance in the outcome (see Appendix C for variance components). Research Question #3: Do the interventions used to increase functional communication skills in children and young adults with autism affect language type (expressive, receptive, and spontaneous)? Of the 43 included studies, 44.19% (N=\9) measured expressive language; 6.98% (7V=3) measured receptive language; and, 48.84% (7V=21) measured spontaneous language. The mean effects of interventions on expressive and spontaneous language were greater than zero (M= 2.530, SE = 0.458), /(40) = 5.522,/? 0.01 and (M= 1.673, SE 0.429), t(40) 3.893,/? 0.01, respectively, with no significant differences. The mean effect of interventions on receptive language was not greater than zero M= 1.447, SE = t(40) = 1.179,/? defining spontaneous language. Language type accounted for 0.55% of the between-study variance in the outcome (see Appendix C for variance components). R 2 2 = ~uNull - if-uComparison} 7uNuli (N=19) N=3) N=21) M = = t(= 5.522,p < M = = = 3.893,p < (M = 1.227), 1.179,p > 0.05, but was not significantly different from that of expressive or spontaneous language. These results are depicted in Figure 2. These findings suggest that language type measured, i.e., the dependent variable, did not significantly impact treatment effectiveness. These results should be interpreted with caution due to the limited number of studies measuring receptive language that met inclusion criteria and due to the lack of consistency among researchers in operationally 58 Mean Effect Size by Language Type 1 • Expressive I I • Spontaneous Language Type Figure 2. Mean Effect Size by Language Type Research Question #4: Does treatment effectiveness vary as a function of language type (expressive, receptive, and spontaneous) and treatment type (total communication, sign only communication, and PECS)? In an attempt to discover additional information about the various forms of language (expressive, receptive, and spontaneous) and their possible influence on treatment effectiveness, the relationship between treatment and language type was explored. Frequencies by treatment and language type are provided in Table 10. Due to an insufficient number of studies, it was not possible to include receptive language in the HLM analyses. Table 11 displays the HLM 6.0 statistics for each treatment when language type was controlled, i.e., expressive and spontaneous language. Although it is inappropriate to make direct comparisons between dependent variables, Figure 3 illustrates the treatment by language type interaction. 3 2.5 2 ~ 1.5 0.5 o lEI Expressive • Receptive El Spontaneous oflanguage PEeS)? 59 Table 10 Frequencies by Treatment and Language Type LANGUAGE TYPE TC SIGN PECS Expressive 12 4 3 Receptive 2 1 0 Spontaneous 3 7 Table 11 Mean Effect Size With Language Type Controlled1 Fixed Effects Coefficient SE T ratio Approximate df p value TC 2.404 0.423 5.685 34 0.000 SIGN 2.159 0.781 2.764 34 0.010 PECS 1.259 0.703 1.790 34 0.082 In light of the above findings, studies measuring expressive and spontaneous language were evaluated independently. The mean effect size for TC studies measuring expressive language was greater than zero (M= 3.108, SE = 0.794), £(16) = 3.916,/? < 0.01. Although the mean effect sizes for Sign and PECS studies measuring expressive language did not reach statistical significance, the mean effect sizes did not differ significantly from that of total communication studies. The mean effect sizes for TC, Sign, and PECS studies measuring spontaneous language were, respectively, greater than zero (M= 1.770, SE = 0.321), £(18) = 5.512,/? < 0.01), (M= 2.697, SE = 0.648), £(18) = 4.159,/? < 0.01), and (M= 1.035, £ £ = 0.445), £(18) = 2.327,/? < 0.05). Additionally, SIGN studies measuring spontaneous language were significantly more effective than 210 11 Controlled* Tratio *does not include Receptive language t(3.916,p differ t(5.512,p t(4.159,p SE = t(= 2.327,p Mean Effect Size by Language Type Figure 3. Treatment X Language Type Interaction PECS studies measuring spontaneous language. Table 12 depicts the differences among treatments when each language type was evaluated independently. Again, caution must be used when interpreting the following results due to a lack of consistency among researchers in defining spontaneous language. The treatment X language type interaction accounted for 1.01% of the variance in effect sizes across studies (see Appendix C for variance components). Research Question #5: Do participant characteristics moderate the effects of interventions used to increase functional communication skills in children and young adults with autism? Each participant variable was run individually to determine if the global effect size was moderated by including the variable at level 1. Table 3 provides summary data for each participant variable. See Appendix C for final estimations of variance 3.5 3 2.5 2 00 Ul 1.5 1 0.5 0 TC SIGN PECS II Expressive • Spontaneous 60 1.01 % Appendi~ effect 61 Table 12 Pair-wise Comparisons by Treatment for Each Language Type Fixed Effects Difference Coefficient T ratio Approximate p value Difference from Expressive TC SIGN -1.353 1.591 -0.850 16 0.408 PECS -1.828 1.762 -1.038 16 0.315 Difference from Expressive SIGN PECS -0.475 2.092 -0.227 16 0.824 from Spontaneous TC SIGN PECS Difference from Spontaneous SIGN PECS 0.926 -0.736 -1.662 0.724 0.549 0.786 1.280 -1.341 -2.114 18 18 18 0.217 0.197 0.049 components, chi-square values, and their respective p values; significant (p < 0.05) chi-square values indicate heterogeneity among studies. Variables that do not have a meaningful zero were grand mean centered and are bolded. R2 i was calculated for each significant variable to determine the proportion reduction in variance or "variance explained" at level 1 (Raudenbush & Bryk, 2002). It should be noted that duration was originally conceptualized as a study characteristic; however, descriptive statistics revealed that participants within the same studies received varying lengths of treatment due to a variety of research design elements and individual outcome measures. As mentioned above, treatment duration was conceptualized as a participant-level variable similar to how one might view hours of homework. Effects SE Tratio df from Ie Difference from Ie 0.186 R2l 62 Research Question #5a: Adaptive Score and Treatment Effectiveness Adaptive scores were reported for 51 of the 138 participants and 12 of the 43 studies. When run as a continuous variable, the mean effect size for participants increased by 0.030 (SE 0.008), £(49) 3.615,/? < 0.01 for each one-point increase in adaptive score. These findings suggest that children and young adults with autism who score higher on standardized adaptive measures experience the greatest change in scores from baseline to treatment phases. Adding adaptive scores, as a predictor of treatment effectiveness, decreased the within study variance by 26.67%. In other words, adaptive scores accounted for over 26% of the participant-level variance in the outcome. Research Question #5b: Age and Treatment Effectiveness Ages were reported for 122 participants in 41 studies. When run as a continuous variable, the mean effect size for participants increased by 0.015 (SE = 0.004), £(120) = 3.908,/? < 0.01 for each additional month in age. These findings suggest that older children with autism experience the greatest change in scores from baseline to treatment phases. Age accounted for 4.49% of the participant-level variance in the outcome. Research Question #5c: Diagnosis and Treatment Effectiveness As Table 3 indicates, there was little variability among participant diagnoses. The mean effect size for individuals diagnosed with autism was greater than zero (M- 2.066, SE = 0.323), £(42) = 6.386,/? < 0.01. Table 13 compares each diagnosis to autism. Caution must be used when interpreting these results due to the limited number of subjects per diagnostic category. In an attempt to gain statistical power, diagnosis was dichotomized as autism (n = 121) and other (n = 17), which included the following diagnoses: PDD, Effectiveness ofthe = t(= 3.615,p Effectiveness = t(= 3.908, p Effectiveness M = = t(= 6.386,p 63 Table 13 Pair-wise Comparisons by Diagnosis Fixed Effects Difference Coefficient SE T ratio Approximate df p value Difference from Autism PDD 0.138 0.719 0.192 133 0.848 Childhood Schizophrenia -0.880 1.416 -0.622 133 0.523 Autistic-like -0.492 0.886 -0.556 133 0.579 Multiple Diagnoses 0.179 0.616 0.292 133 0.771 Other (PDD, Childhood Schizophrenia, Autistic-like, Multiple Diagnoses) -0.014 0.429 -0.033 136 0.974 childhood schizophrenia, autistic-like, and multiple diagnoses. Similarly, the mean effect sizes for the dichotomized categories (autism and other) did not differ significantly from each other (see Table 13). Research Question #5d: Gender and Treatment Effectiveness Gender was reported for 116 participants of whom 19 were female and 97 were male. The mean effect size for males was greater than zero (M= 2.091, SE 0.312), £(41) 6.691, p 0.01 and did not differ significantly from females (Mdiff= -0.120, SE = 0.331), t(l 14) = -0.362, p = 0.718. Research Question #5e: IQ Score and Treatment Effectiveness IQ standard scores were reported for 52 participants in 13 studies. When run as a continuous variable, the mean effect size for participants increased by 0.028 (SE 0.013), £(50) 2.187,/? = 0.033 for each one-point increase in IQ score. These findings Effects Tratio bifference Table13). Effectiveness M = = t(= < Mdiff= SE= 114) = = Effectiveness = t(50) = 2.187,p 64 Pair-wise Comparisons by Cognitive Classification Fixed Effects Difference Coefficient SE T ratio Approximate p value Difference from Moderate MR Normal (70 < IQ) -0.742 1.557 -0.477 58 0.635 Mild MR (50<IQ<70) 0.349 0.501 0.697 58 0.488 Severe MR (20<IQ<35) -0.502 0.423 -1.187 58 0.240 Profound MR (IQ < 20) -0.960 0.644 -1.491 58 0.141 suggest that children and young adults who score higher on standardized intelligence measures experience the greatest change in scores from baseline to treatment phases. IQ accounted for 12.13% of the participant-level variance in the outcome. Research Question #5f: MR Category and Treatment Effectiveness A cognitive classification could be assigned to 63 participants. Table 3 displays the number of individuals by category. Question #5e analyzed IQ score as a continuous variable resulting in a significant main effect. An additional 11 participants were included when analyzing IQ scores by categories, although a less powerful statistical procedure. The mean effect size for individuals classified with moderate mental retardation (35 < IQ < 50) was greater than zero (M = 2.256, SE = 0.520), t(\9),p < 0.01. Table 14 compares each classification to moderate mental retardation. Caution must be used when interpreting these results due to the limited number of participants per classification. In an attempt to gain statistical power, cognitive classification was Table 14 Effectiveness S M= t(19),p Classification Effects Tratio df S (50 S IQ < 70) (20 S IQ < 65 dichotomized as severe/profound, or IQ < 35, (n 30), and mild/moderate, or IQ > 35, (n 33). Although the mean effect size for IQ < 35 was greater than zero (M 1.698, SE = 0.497), £(19) = 3.416,/? < 0.01, as expected, the difference in mean effect size for IQ > 35 did not meet statistical significance (Mdiff= 0.659, SE 0.356), £(61) 1.853,/? > 0.05. As discussed previously, IQ does not have a meaningful zero requiring that analyses be centered about the grand mean for all participants, which was equal to 38.05. Analyses by cognitive classification may not be appropriate for this meta-analysis. Results from question #5e should be utilized when examining the impact that an individual's IQ may have on treatment effectiveness. Research Question #5g: Receptive Language Score and Treatment Effectiveness Receptive language standard scores were reported for 70 participants in 13 studies. When run as a continuous variable, the mean effect size for participants increased by 0.015 (SE = 0.007), £(68) = 2.258,/? = 0.027 for each one point increase in receptive language score. These findings suggest that children and young adults who score higher on receptive language standardized measures experience the greatest change in scores from baseline to treatment phases. Receptive language scores accounted for 2.65% of the participant-level variance in the outcome. Research Question #5h: Nonverbal/Verbal Classification and Treatment Effectiveness Verbal classifications (nonverbal/verbal/echolalic) were reported for 123 participants in 41 studies. The mean effect size for individuals with a nonverbal classification was greater than zero (M= 2.453, SE 0.354), £(40) 6.916,/? < 0.01. A verbal classification resulted in a mean effect size of 1.624 (SE 0.412), £(40) 3.942,/? < 0.01, which indicated a significant difference in treatment magnitude from a nonverbal = ~ = (M = SE = t(= 3.416,p ~ significance. (Mdif/= SE = t(61) = 1.853,p centered about the grand mean for all participants, which was equal to 38.05. Analyses by cognitive classification may not be appropriate for this meta-analysis. Results from have on treatment effectiveness. Effectiveness t(2.258,p NonverbalNerbal Effectiveness nonverballverbal/M = = t(= 6.916,p = t(= 3.942,p 66 classification (see Table 15). The mean effect size for individuals classified as echolalic was not statistically greater than zero (M = 1.203, SE = 0. 756), £(40) = 1.590,p = 0.119, and difference scores did not reach statistical significance (see Table 15). Figure 4 visually depicts the mean effect size for each verbal classification. To further explore these differences, verbal classification was dichotomized (nonverbal/verbal). Individuals who had been described as echolalic were recategorized as verbal, which is consistent with the operational definition for a verbal classification. The resulting mean effect size for individuals with a nonverbal classification was 2.445 (SE 0.351), £(40) 6.953, p < 0.01, which was significantly larger than the mean effect size of 1.555 for individuals classified as verbal (Mdiff= -0.890, SE = 0.401), £(121) - 2.219, p = 0.028. Figure 5 graphically displays this main effect. These findings suggest that children and young adults with autism who are nonverbal experience the greatest change in scores from baseline to treatment. It should be noted that baseline scores often indicated a frequency of zero or zero percent correct, thus requiring effect sizes to be calculated from the treatment standard deviation rather Table 15 Pair-wise Comparisons by Verbal Classification Fixed Effects Difference SE T ratio Approximate /? value Coefficient df Difference from Nonverbal Verbal -0.828 0.417 -1.985 120 0.049 -1.249 0.773 -1.616 120 0.108 Difference from Verbal Echolalic -0.421 0.771 -0.547 120 0.585 M= = 0.756), t(= = classification. = t(= MdifJ= = t(121) = effect. Classification Tratio p Echolalic Verbal 67 3 1-5 Mean Effect Size by Verbal Classification Nonverbal Figure 4. Mean Effect Size by Verbal Classification Figure 5. Mean Effect Size by Nonverbal and Verbal Classification 3 2.5 2 00 ~ 1.5 0.5 o Verbal Echolalic Classification Mean Effect Size by Nonverbal and Verbal Classification 3 2.5 2 r:g 1.5 0.5 o Nonverbal Verbal Classification 68 than a pooled standard deviation. Participants with a nonverbal classification are most likely to have baseline scores equal to zero, possibly inflating the true treatment effect size. Nonverbal/Verbal classification accounted for 2.20% of the participant-level variance in the outcome. Research Question #5i: Duration and Treatment Effectiveness Duration in weeks was reported for 137 participants in 42 studies. Participants within the same studies received varying lengths of treatment due to a variety of research design elements and individual outcome measures. For purposes of the present meta­analysis, treatment duration was conceptualized as a participant-level variable similar to how one might view hours of homework (Roberts, 2005). When run as a continuous variable, the mean effect size for participants increased by 0.008 (SE = 0.002), £(135) = 3.938,/? < 0.01) for each additional week of treatment. These findings suggest that children and young adults with autism continue to benefit from functional communication interventions as treatment duration increases. Treatment duration accounted for 15.15% of the participant-level variance in the outcome. Since duration was extremely variable among participants and studies (see Table 4), age was added to this level 1 model in an attempt to parse out treatment gains that might be better explained by age and/or maturation than true treatment duration. Figure 6 illustrates the impact of both duration and maturation on effect size. Briefly, the line labeled "Duration" depicts the effects of holding age constant at 23 months and varying duration from 2 to 171 weeks. The line labeled "Age" illustrates the effects of holding duration constant at 2 weeks and varying age from 23 to 192 months. Adding both age effect Effectiveness = t(135) = 3.938,p oftreatment. functional 69 Mean Effect Size by Age and Duration -•- Age (duration constant) *- (age Age in months Duration in weeks Figure 6. Effect of Duration and Maturation on Global Effect Size and duration, as predictors of treatment effectiveness, decreased the within study variance by 17.90% (see Appendix C for variance components). Research Question #6: Are there interaction effects between coded participant variables (adaptive score, age, diagnosis, gender, IQ, MR category, receptive language score, verbal classification, and duration*) and each intervention type (total communication, sign only communication, and PECS) used to increase functional communication skills in children and young adults with autism? As previously noted, several of the participant variables resulted in positive main effects, i.e., older individuals, those exposed to additional weeks of intervention, and individuals with higher adaptive, IQ, and receptive language scores evidenced greater gains from baseline to treatment phases. In order to determine if treatment types were differentially impacted by participant variables, each interaction was run individually. Appendix C displays the variance components, chi-square values, and their respective p 4 3.5 3 2.5 rF.l 2 ~ 1.5 1 0.5 0 Effuct -+-A~ ___ Duration constant) duration) 70 p values; variables that do not have a meaningful zero were grand mean centered and are bolded. R2 i was calculated for each significant interaction to determine the proportion reduction in variance. Each significant interaction will be discussed individually. Research Question #6: Adaptive Score X Treatment Interaction In order to examine possible cross-level interactions, adaptive score was run as a continuous variable at level 1 with treatment at level 2. The mean effect size for participants who received total communication interventions increased by 0.040 (SE = 0.009), t(47) = 4.338,/? < 0.01 for each one point increase in adaptive score. Similar to total communication, the mean effect size for PECS increased by 0.004 with each one point increase in adaptive score (Mdiff= -0.036, SE = 0.019), £(47) = -1.856,/? = 0.069, indicating no significant difference from total communication. In contrast, the mean effect size for sign only communication decreased by 0.059 with each one point increase in adaptive score (Mdiff= -0.099, SE = 0.046), £(47) = -2.118,/? = 0.039. Figure 7 visually represents how treatment type varied significantly as a function of adaptive scores. Treatment accounted for 30.83% of the reliable variance among studies in the effects of adaptive scores on outcomes. Research Question #6: IQ X Treatment Interaction In order to examine possible cross-level interactions, IQ score was run as a continuous variable at level 1 with treatment at level 2. The mean effect size for participants who received total communication interventions increased by 0.055 (SE = 0.015), £(48) = 3.676,p < 0.01 for each one point increase in IQ. Similar to total communication, the mean effect size for PECS increased by 0.011 with each one point increase in adaptive score (A%= -0.044, SE = 0.033), £(48) = -1.296,/? = 0.202. In R2j = = 4.338,p Mdiff= t(1.856,p Mdiff= t( 2.118, p 10 = t(= Mdiff= t(1.296,p 71 Adaptive Score X Treatment Interaction Mean Adaptive Score = 29.16 Figure 7. Adaptive Score X Treatment Interaction contrast, the mean effect size for sign only communication decreased by 0.034 with each one point increase in IQ (Mdiff= -0.089, SE 0.030), £(48) -2.978, p < 0.01. Figure 8 visually represents how treatment type varied significantly as a function of IQ score. These findings illustrate the variable impact that participant IQ may have on treatment effectiveness. Similar to the main effect that IQ had on the global ES, children and young adults with autism who scored higher on standardized intelligence measures showed the greatest improvement in scores from baseline to treatment when receiving TC and PECS interventions. In contrast, children and young adults with autism who scored higher on standardized intelligence measures evidenced the smallest treatment gains when receiving SIGN interventions. Treatment accounted for 27.83% of the reliable variance among studies in the effects of IQ on outcomes. 3 2.5 2 1.5 -+--TC* ru/.1J 0.5 ----SIGN* 0 -IJr-- PECS -0.5 -1 -1.5 -2 MdijJ= = t( = P ofIQ ofIQ 72 IQ X Treatment Interaction -•-TC* *-SIGN* PECS 38.05 70 Mean IQ Score = 38.05 Figure 8. IQ X Treatment Interaction Research Question #7: Do study characteristics moderate the effects of interv