Superheroes social skills: a study examining the effects of using an evidence-based approach to teach social skills to children with high-incidence disabilities

The current study evaluated the use of a manualized social skills program, Superheroes Social Skills, to increase the use of prosocial behaviors and decrease the use of aggressive behaviors for children with externalizing behaviors. The training was implemented by a school psychologist in a pullout group with four children with highincidence disabilities and four typically developing peers at a public elementary school. The program implemented was a multimedia, high interest program that incorporates the use of animation to teach the lessons. This program also uses evidence-based practices (e.g., video-modeling, peer mediation, social stories, and self-management) to help increase the effectiveness of the intervention. There were eleven skills taught during one 30-minute session. Two sessions were taught per week. Children were videotaped during free play and recess observations for baseline, treatment, and follow-up. Their behaviors were then coded using a partial interval recording system. The behaviors observed were verbal aggression, physical aggression, neutral behavior, positive initiations, and positive responses. The observation codes were used to calculate effect sizes, percentage of nonoverlapping data points, and percentage of all nonoverlapping data points. The Social Skills Improvement System (SSIS) was used as a pre- and postmeasure of treatment effectiveness. Measures were also used to determine social validity, consumer satisfaction, and treatment integrity. These measures were analyzed using descriptive statistics. Results indicated that this intervention was effective for decreasing aggressive behaviors, decreasing neutral play, and increasing positive responses in both the treatment setting and the generalized recess setting. Results were also maintained at a 2-week follow-up. Parents and participants indicated the program was effective and favorable. The results of the SSIS indicated minimal treatment effect, although teachers rated a significant increase in social skills. The treatment was implemented with high treatment fidelity. Overall, this study found that the Superheroes Social Skills Program was an effective intervention for children with high-incidence disabilities and externalizing behaviors.

SUPERHEROES SOCIAL SKILLS: A STUDY EXAMINING THE EFFECTS OF USING AN EVIDENCE-BASED APPROACH TO TEACH SOCIAL SKILLS TO CHILDREN WITH HIGH-INCIDENCE DISABILITIES by Julia Ann Kelly Hood A dissertation submitted to the faculty of The University of Utah in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Educational Psychology The University of Utah December 2011 Copyright ! Julia Ann Kelly Hood 2011 All Rights Reserved Th e Un i v e r s i t y o f Ut a h Gr a d u a t e S c h o o l STATEMENT OF DISSERTATION APPROVAL The dissertation of Julia Ann Kelly Hood has been approved by the following supervisory committee members: William Jenson , Chair 10/26/2011 Date Approved Elaine Clark , Member 10/26/2011 Date Approved Dan Olympia , Member 10/26/2011 Date Approved Robert O'Neill , Member 10/26/2011 Date Approved Julie Bowen , Member Date Approved and by Elaine Clark , Chair of the Department of Educational Psychology and by Charles A. Wight, Dean of The Graduate School. ! ! ! ! ABSTRACT The current study evaluated the use of a manualized social skills program, Superheroes Social Skills, to increase the use of prosocial behaviors and decrease the use of aggressive behaviors for children with externalizing behaviors. The training was implemented by a school psychologist in a pullout group with four children with high-incidence disabilities and four typically developing peers at a public elementary school. The program implemented was a multimedia, high interest program that incorporates the use of animation to teach the lessons. This program also uses evidence-based practices (e.g., video-modeling, peer mediation, social stories, and self-management) to help increase the effectiveness of the intervention. There were eleven skills taught during one 30-minute session. Two sessions were taught per week. Children were videotaped during free play and recess observations for baseline, treatment, and follow-up. Their behaviors were then coded using a partial interval recording system. The behaviors observed were verbal aggression, physical aggression, neutral behavior, positive initiations, and positive responses. The observation codes were used to calculate effect sizes, percentage of nonoverlapping data points, and percentage of all nonoverlapping data points. The Social Skills Improvement System (SSIS) was used as a pre- and post-measure of treatment effectiveness. Measures were also used to determine social ! ! "#! validity, consumer satisfaction, and treatment integrity. These measures were analyzed using descriptive statistics. Results indicated that this intervention was effective for decreasing aggressive behaviors, decreasing neutral play, and increasing positive responses in both the treatment setting and the generalized recess setting. Results were also maintained at a 2- week follow-up. Parents and participants indicated the program was effective and favorable. The results of the SSIS indicated minimal treatment effect, although teachers rated a significant increase in social skills. The treatment was implemented with high treatment fidelity. Overall, this study found that the Superheroes Social Skills Program was an effective intervention for children with high-incidence disabilities and externalizing behaviors. ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! "#!$%!&$&'()*!&)+!,-..#/0(12!3&$(4%!! 5602127!8&+7!9#$7!"#$7!6-,&)7!9(:27!60244&7!62&)7!;(<#4427!=44#/&7!&)+!>2)/%?@!!! A!<#-4+)B0!C&12!+#)2!0C(,!D(0C#-0!%#-@! E4,#!0#!$%!./#32,,#/,!&)+!3244#D!,0-+2)0,!DC#!<C&442)*2!$2!&)+!(),.(/2!$2!+&(4%@! ! ! ! ! ! "#$%&!'(!)'*"&*"+! ! ! ! #$+",#)"---------------------------------------...! ! %/+"!'(!"#$%&+------------------------------------.0! ! %/+"!'(!(/12,&+-----------------------------------330.! ! )456789! ! ! :3 /*",';2)"/'*-----333--------------------------33:! ! +<=.5>!+?.>>@!5AB!+<=.5>!)<C6878A=8-----------------333D! )<CC<A!)<C6<A8A7@!<E!+<=.5>!+?.>>@!F9<G95C@----------3333H! )I998A7!+<=.5>!+?.>>@!F9<G95C@--------------------3::! )I998A7!+<=.5>!+?.>>@!F9<G95C@!E<9!)4.>B98A!J.74! &0789A5>.K.AG!$845L.<9@-----------------------333:D! &L.B8A=8M$5@8B!F95=7.=8-----------------------33:N! O875M#A5>P@8@----------------------------3333QD! &EE.=5=P!<E!+<=.5>!+?.>>@!F9<G95C@------------------333QR! 18A895>.K57.<A!<E!+<=.5>!+?.>>@--------------------333DQ! &A45A=.AG!18A895>.K57.<A-----------------------DH! F9<C.@.AG!#669<5=48@!7<!O5?8!+<=.5>!+?.>>@!! O<98!&EE8=7.L8-----------------------------3DS! ! T.B8<!O<B8>.AG!/A789L8A7.<A@----------------33DS! ! F889!O8B.578B!/A789L8A7.<A@-----------------3DU! ! +8>EMO5A5G8C8A7!/A789L8A7.<A@---------------33R:! ! +<=.5>!+7<9.8@-------------------------333RQ! ! /A=985@.AG!18A895>.K57.<A!J.74!V+7.=?.A8@@W---------RD! +I689489<!+<=.5>!+?.>>@------------------------33RX! +ICC59P--------------------------------HY! +7578C8A7!<E!FI96<@8-------------------------3H:! ,8@859=4!ZI8@7.<A@--------------------------3HQ! ! ! Q3 O&"[';+-----------------------------------33HR! ! F597.=.65A7@------------------------------33HH! ! L..! +877.AG---------------------------------33XY! ;868AB8A7!O85@I98@----------------------3---3X:! ! '\@89L57.<A!+P@78C----------------------X:! ! +<=.5>!+?.>>@!/C69<L8C8A7!+P@78C--------------3XD! ! $845L.<9!/A789L8A7.<A!,57.AG!+=5>8-------------33XR! ! +<=.5>!T5>.B.7P!)48=?>.@7-------------------333XR! ! )4.>B!)<A@IC89!+57.@E5=7.<A!+I9L8P-------------3XH! ! "9857C8A7!(.B8>.7P!)48=?>.@7-----------------3XH! ;8@.GA---------------------------------33XX! F9<=8BI98@-------------------------------XS! $5@8>.A8--------------------------------333XU! "9857C8A7-------------------------------33NY! '7489!O85@I98@----------------------------33NQ! (<>><JMI6-------------------------------333NQ! ;575!#A5>P@.@------------------------------NQ! ! ! D3 ,&+2%"+------------------------------------3NS! ! "9857C8A7!/A78G9.7P--------------------------3NU! ,8>.5\.>.7P!<E!'\@89L57.<A@----------------------33NU! ,8@859=4!ZI8@7.<A!:--------------------------SY! ,8@859=4!ZI8@7.<A!Q--------------------------US! ,8@859=4!ZI8@7.<A!D-------------------------33:::! ,8@859=4!ZI8@7.<A!R-------------------------33:DR! ,8@859=4!ZI8@7.<A!H-------------------------33:DN! ,8@859=4!ZI8@7.<A!X-------------------------33:RU! ,8@859=4!ZI8@7.<A!N-------------------------33:HQ! ,8@859=4!ZI8@7.<A!S-------------------------33:HH! ! ! R3 ;/+)2++/'*----------------------------------:HN! ! O5.A!(.AB.AG@-----------------------------3:HU! F#*;!5AB!)<48A]@!!-------------------------333:XY! "9857C8A7!&EE8=7!+.K8!$5@8B!<A!;575!#A5>P@.@!O874<B@------33:X:! +<=.5>!+?.>>@!/C69<L8C8A7!+P@78C------------------3:XQ! F9<G98@@!O<A.7<9.AG-------------------------333:XD! )<A@IC89!+57.@E5=7.<A!5AB!+<=.5>!T5>.B.7P--------------:XR! "9857C8A7!/A78G9.7P--------------------------3:XH! [<J!,8@I>7@!)<998@6<AB!J.74!F9.<9!,8@859=4----------333-:XH! %.C.757.<A@!5AB!(I7I98!,8@859=4-------------------:NR! /C6>.=57.<A@!E<9!F95=7.=8-----------------------33:NX! ! ! ! ! L...! #668AB.=8@! ! #3 +#OF%&!%&++'*-------------------------------:NS! ! $3 '$+&,T#"/'*!)';/*1!+^+"&O----------------------3:S:! ! )3! $&[#T/',!/*"&,T&*"/'*!,#"/*1!+)#%&----------------:SR! ! ;3!!! +')/#%!T#%/;/"^!+)#%&--------------------------33:SN! ! &3! )[/%;!)'*+2O&,!+#"/+(#)"/'*!+2,T&^----------------:SU! ! (3!!! F#,&*"#%!F&,O/++/'*--------------------------33:U:! ! 13!!! )[/%;!#++&*"--------------------------------:UN! ! [3! F%#)&O&*"!)[&)_%/+"--------------------------3QY:! ! /3! ",&#"O&*"!(/;&%/"^!)[&)_%/+"--------------------3QYR! ! `3! /*;/T/;2#%!F#,"/)/F#*"!1,#F[+!(',!#*#%'1!",&#"O&*"----QYX! ! _3! /*;/T/;2#%!F#,"/)/F#*"!1,#F[+!(',!,&)&++!",&#"O&*"----33QQ:! ! %3!!! /*;/T/;2#%!F#,"/)/F#*"!1,#F[+!(',!#*#%'1!('%%'aM2F----3QDX! ! O3!!! /*;/T/;2#%!F#,"/)/F#*"!1,#F[+!(',!,&)&++!('%%'aM2F----333QH:! ! ,&(&,&*)&+------------------------------------33QXX! ! ! ! ! ! "#$%!&'!%()"*$! ! ! ! +,!!%-./0/01!$2342052!67!869960!$65/.:!$;/::<!=-61-.9<>>>>>>>>>>>>>>?! ! @,!!$;/::<!%.41AB!#0!869960!$65/.:!$;/::<!=-61-.9<>>>>>>>>>>>>>>>>>,C! ! D,!!869960!$65/.:!$;/::<!=-61-.9<>>>>>>>>>>>>>>>>>>>>>>>>>>+@! ! E,!!869960!$65/.:!$;/::<!=-61-.9<!76-!*FB2-0.:/G/01!)2A.H/6-<>>>>>>>>>>,+E! ! I,!!=.-B/5/J.0B!8A.-.5B2-/<B/5<>>>>>>>>>>>>>>>>>>>>>>>>>>>>>IK! ! ?,!!(H2-.12!=.-B/5/J.0B!*01.12920B!/0!)2A.H/6-<>>>>>>>>>>>>>>>>>>LK! ! K,!!(0.:61!&M<2-H.B/60!%-2.B920B!N2<4:B<>>>>>>>>>>>>>>>>>>>>>>LL! ! L,!!=.-B/5/J.0B!+!*01.12920B!/0!)2A.H/6-<>>>>>>>>>>>>>>>>>>>>>>CO! ! C,!!=.-B/5/J.0B!@!*01.12920B!/0!)2A.H/6-<>>>>>>>>>>>>>>>>>>>>>,,,C@! ! +O,!!=.-B/5/J.0B!D!*01.12920B!/0!)2A.H/6-<>>>>>>>>>>>>>>>>>>>>>CE! ! ++,!!=.-B/5/J.0B!E!*01.12920B!/0!)2A.H/6-<>>>>>>>>>>>>>>>>>>>>>C?! ! +@,!!N252<<!&M<2-H.B/60!%-2.B920B!N2<4:B<>>>>>>>>>>>>>>>>>>>>,,+O@! ! +D,!!(0.:61!&M<2-H.B/60!'6::6PQ4J!N2<4:B<>>>>>>>>>>>>>>>>>>>>,,,++K! ! +E,!!N252<<!&M<2-H.B/60!'6::6PQ4J!N2<4:B<>>>>>>>>>>>>>>>>>>>>,,,+@C! ! +I,!!(H2-.12!N.B/01<!60!BA2!)2A.H/6-!#0B2-H20B/60!N.B/01!$5.:2>>>>>>>>>,+DI! ! +?,!!(H2-.12!=.-B/5/J.0B!$56-2<!60!BA2!$$#$>>>>>>>>>>>>>>>>>>>>,,+EO! ! +K,!!$$#$!$56-2<R!=.-B/5/J.0B!+>>>>>>>>>>>>>>>>>>>>>>>>>>>>+E@! ! +L,!!$$#$!$56-2<R!=.-B/5/J.0B!@>>>>>>>>>>>>>>>>>>>>>>>>>>>>+EI! ! +C,!!$$#$!$56-2<R!=.-B/5/J.0B!D>>>>>>>>>>>>>>>>>>>>>>>>>>>>+EK! ! F! ! @O,!!$$#$!$56-2<R!=.-B/5/J.0B!E>>>>>>>>>>>>>>>>>>>>>>>>>>>,,+IO! ! @+,!!$65/.:!S.:/T/BU!$5.:2!$56-2<>>>>>>>>>>>>>>>>>>>>>>>>>>,,+I+! ! @@,!!8A/:T!860<492-!$.B/<7.5B/60!$4-H2U!$56-2<>>>>>>>>>>>>>>>>>,,+IE! ! ! ! ! ! "#$%!&'!'#()*+$! ! ! ! ,- +./0123!45!/!6789!:!.!:!%/;23<<<<<<<<<<<<<<<<<<<<<<<<<-==! ! :-!7>3?/@3!7A/24@!B3/CD?3!45!E3?;/2!7@@?3CCF4A!54?!/22!6/?GFHF1/AGC<<<<-<<I:! ! J-!7>3?/@3!7A/24@!B3/CD?3!45!6KLCFH/2!7@@?3CCF4A!54?!/22!6/?GFHF1/AGC<<<<<-I:! ! M-!7>3?/@3!7A/24@!B3/CD?3!45!!7@@?3CCF4A!54?!/22!6/?GFHF1/AGC<<<<<<<<<<IJ! ! ! N-!!7>3?/@3!7A/24@!B3/CD?3!45!64CFGF>3!#AFGF/GF4AC!54?!/22!6/?GFHF1/AGC<<<<<---IJ! ! O-!7>3?/@3!7A/24@!B3/CD?3!45!64CFGF>3!*3C14AC3C!54?!/22!6/?GFHF1/AGC<<<<<<IM! ! =-!7>3?/@3!7A/24@!B3/CD?3!45!$4HF/2!+A@/@303AG!54?!/22!6/?GFHF1/AGC<<<<<<IN! ! ! I-!7>3?/@3!7A/24@!B3/CD?3!45!83DG?/2!62/L!54?!/22!6/?GFHF1/AGC<<<<<<<<<--IO! ! P-!7>3?/@3!*3H3CC!B3/CD?3!45!E3?;/2!7@@?3CCF4A!54?!/22!6/?GFHF1/AGC<<<<<--,QQ! ! ,Q-!7>3?/@3!*3H3CC!B3/CD?3!45!6KLCFH/2!7@@?3CCF4A!54?!/22!6/?GFHF1/AGC<<<<,QQ! ! ,,-!7>3?/@3!*3H3CC!B3/CD?3!45!!7@@?3CCF4A!54?!/22!6/?GFHF1/AGC<<<<<<<<--,Q,! ! ! ,:-!!7>3?/@3!*3H3CC!B3/CD?3!45!64CFGF>3!#AFGF/GF4AC!54?!/22!6/?GFHF1/AGC<<<<-,QM! ! ,J-!7>3?/@3!*3H3CC!B3/CD?3!45!64CFGF>3!*3C14AC3C!54?!/22!6/?GFHF1/AGC<<<<-,QM! ! ,M-!7>3?/@3!*3H3CC!B3/CD?3!45!$4HF/2!+A@/@303AG!54?!/22!6/?GFHF1/AGC<<<<--,QN! ! ! ,N-!7>3?/@3!*3H3CC!B3/CD?3!45!83DG?/2!62/L!54?!/22!6/?GFHF1/AGC<<<<<<<<,QN! ! ,O-!!7>3?/@3!'4224RSD1!7A/24@!B3/CD?3!45!E3?;/2!7@@?3CCF4A!54?!/22!! ! ! 6/?GFHF1/AGC<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<---,,:! ! ,=-!7>3?/@3!'4224RSD1!7A/24@!B3/CD?3!45!6KLCFH/2!7@@?3CCF4A!54?!/22!! ! ! 6/?GFHF1/AGC ----------------------------------------------------------------------------------------------------------------,,J! ! ! ! .FF! ,I-!7>3?/@3!'4224RSD1!7A/24@!B3/CD?3!45!!7@@?3CCF4A!54?!/22!! ! ! 6/?GFHF1/AGC ----------------------------------------------------------------------------------------------------------------,,J! ! ! ,P-!!7>3?/@3!'4224RSD1!7A/24@!B3/CD?3!45!64CFGF>3!#AFGF/GF4AC!54?!/22!! ! ! 6/?GFHF1/AGC ----------------------------------------------------------------------------------------------------------------,,N! ! :Q-!7>3?/@3!'4224RSD1!7A/24@!B3/CD?3!45!64CFGF>3!*3C14AC3C!54?!/22!! ! ! 6/?GFHF1/AGC ----------------------------------------------------------------------------------------------------------------,,N! ! :,-!7>3?/@3!'4224RSD1!7A/24@!B3/CD?3!45!$4HF/2!+A@/@303AG!54?!/22!! ! ! 6/?GFHF1/AGC ----------------------------------------------------------------------------------------------------------------,,O! ! ! ::-!7>3?/@3!'4224RSD1!7A/24@!B3/CD?3!45!83DG?/2!62/L!54?!/22!! ! ! 6/?GFHF1/AGC ----------------------------------------------------------------------------------------------------------------,,O! ! :J-!7>3?/@3!'4224RSD1!*3H3CC!B3/CD?3!45!E3?;/2!7@@?3CCF4A!54?!/22!! ! ! 6/?GFHF1/AGC -----------------------------------------------------------------------------------------------------------<-,:M! ! :M-!7>3?/@3!'4224RSD1!*3H3CC!B3/CD?3!45!6KLCFH/2!7@@?3CCF4A!54?!/22!! ! ! 6/?GFHF1/AGC ----------------------------------------------------------------------------------------------------------------,:M! ! :N-!7>3?/@3!'4224RSD1!*3H3CC!B3/CD?3!45!!7@@?3CCF4A!54?!/22!! ! ! 6/?GFHF1/AGC ----------------------------------------------------------------------------------------------------------------,:N! ! ! :O-!!7>3?/@3!'4224RSD1!*3H3CC!B3/CD?3!45!64CFGF>3!#AFGF/GF4AC!54?!/22!! ! ! 6/?GFHF1/AGC ----------------------------------------------------------------------------------------------------------------,:N! ! :=-!7>3?/@3!'4224RSD1!*3H3CC!B3/CD?3!45!64CFGF>3!*3C14AC3C!54?!/22!! ! ! 6/?GFHF1/AGC ----------------------------------------------------------------------------------------------------------------,:O! ! :I-!7>3?/@3!'4224RSD1!*3H3CC!B3/CD?3!45!$4HF/2!+A@/@303AG!54?!/22!! ! ! 6/?GFHF1/AGC ----------------------------------------------------------------------------------------------------------------,:O! ! ! :P-!7>3?/@3!'4224RSD1!*3H3CC!B3/CD?3!45!83DG?/2!62/L!54?!/22!! ! ! 6/?GFHF1/AGC ----------------------------------------------------------------------------------------------------------------,:I! ! JQ-!!6/?3AG!*/GFA@C!4A!GK3!$$#$T!6/?GFHF1/AG!,<<<<<<<<<<<<<<<<<<--,MQ! ! J,-!!%3/HK3?!*/GFA@C!4A!GK3!$$#$!6/?GFHF1/AG!,<<<<<<<<<<<<<<<<<<,M:! ! J:-!6/?3AG!*/GFA@C!4A!GK3!$$#$T!6/?GFHF1/AG!:<<<<<<<<<<<<<<<<<<<,MJ! ! JJ-!!%3/HK3?!*/GFA@C!4A!GK3!$$#$!6/?GFHF1/AG!:<<<<<<<<<<<<<<<<<<,MM! ! JM-!6/?3AG!*/GFA@C!4A!GK3!$$#$T!6/?GFHF1/AG!J<<<<<<<<<<<<<<<<<<<,MN! ! ! .FFF! JN-!!%3/HK3?!*/GFA@C!4A!GK3!$$#$!6/?GFHF1/AG!J<<<<<<<<<<<<<<<<<<,M=! ! JO-!6/?3AG!*/GFA@C!4A!GK3!$$#$T!6/?GFHF1/AG!M<<<<<<<<<<<<<<<<--<<-,MI! ! J=-!!%3/HK3?!*/GFA@C!4A!GK3!$$#$!6/?GFHF1/AG!M<<<<<<<<<<<<<<<<--<--,MP! ! CHAPTER 1 INTRODUCTION Many students within the public schools system display deficits that impact their educational performance and social relationships with peers and adults. Many of these students are classified with a high-incidence disability and receive special education services aimed at helping them succeed academically. The term high-incidence disabilities refers to four specific disabilities that are among the most common special education classifications and account for more than 70% of all students who are served within special education (Sabornie, Evans, & Cullinan, 2006). High-incidence disabilities include learning disability (LD), speech/language impairment (SLI), emotional-behavioral disorder (EBD), and mild intellectual disability (MID). While most children with high-incidence disabilities experience social deficits and behavior deficits, Sabornie, Cullinan, Osborne, and Brock (2005) found that children with EBD were reported to exhibit more problem behavior than approximately 76% of the students with LD or MID. This research implies that there is a great need for effective social skills training for children with EBD. Of these students with EBD, there are the subclassifications of internalizing disorders and externalizing disorders. Externalizing disorders is a broad category that encompasses a range of specific disorders. Externalizing disorders, such as conduct disorder and Attention ! "! Deficit/Hyperactivity Disorder (ADHD), differ from internalizing disorders, such as depression or anxiety, in that the behavior is manifested outwardly and visibly through aggression and/or disruptive behavior. The Utah State Office of Education (2007) defines externalizing disorders as "behavioral problems that are directed outwardly by the student toward the social environment, and usually involve behavioral excesses" (p. 37) ad internalizing disorders are defined as "a class of behavior problems that are directed inwardly, and often involve behavioral deficits" (p. 37). Within the schools, children often receive an educational classification of Emotional Disturbance (ED), Specific Learning Disability (SLD), Speech/Language Impairment (SLI), or Other Health Impaired (OHI). The Utah State Office of Education (2007) has provided educators with specific definitions of these four disorders in their Special Education Rules. Some of the psychiatric disorders that are considered to be in these educational classification categories include Depression, Speech/Language Impairment, Anxiety Disorder, Learning Disability, Conduct Disorder, and ADHD. The Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR; American Psychiatric Association, 2000) provides criteria for diagnosing these disorders. While many children with high-incidence disabilities experience difficulty in exhibiting pro-social behaviors, children with externalizing disorders are disruptive in the classroom and can more outwardly alienate peers with their behavior. Conversely, children with internalizing disorders may have emotional arousals that interfere with their ability to perform the pro-social behaviors. This can have an effect on their academic performance, as well as their peer relationships. ! #! Social Skills and Social Competence Greenspan (1980) posits that social skills are the key to being able to relate to others normally. The level of a person's social development is what allows us to appear normal, not our ability to read and write. Social skills are an essential element of development, but many children do not naturally acquire social skills. This puts them at risk for detrimental effects on their peer relationships, adult interactions, school performance, and work relationships. A lack of social skills can lead to poor peer relationships and rejection by others. Ultimately, this can impact the child's emotional well-being (Morgan & Jenson, 1988). It is important to provide social skills training to children who do not naturally acquire these skills in order to counteract the potential detrimental effects on their relationships and their emotional well-being. Morgan and Jenson (1988) define social skills as "verbal and nonverbal behaviors a person uses to interact with others so that the encounter is mutually beneficial and reinforcing" (p. 245). These basic skills, both verbal and nonverbal, are essential to facilitate positive interactions with others. While social skills can be defined and taught to children, many fail to apply these social behaviors to social interactions and the development of peer relationships. Gresham and Elliott (1987) define the difference between four types of deficits that can affect the ability to learn and use social skills in generalized settings. Skill deficits, which are often called acquisition deficits, are present when the child lacks the ability, or has never been taught to, interact in a way that is socially appropriate. Performance deficits are present when the specific skills have been learned, but the individual is ! $! unable to perform the skill, which may be due to a lack of motivation or a lack of stimulus control. Self-control skill deficits occur when the child is unable to learn the social skill due to an emotional arousal that blocks learning or use of this skill. Self-control performance deficits occur when the child has learned a social skill, but is unable to perform the skill because of an emotional arousal and inability to modulate this arousal. There are many factors that affect a child's ability to become socially competent, which is why teaching the skills alone is not sufficient for the child to build and maintain social relationships. The integration of the basic and specific learned social skills during social interactions can be defined as social competence. Gresham and Elliott (1987) consider social competence to be the combined use of social skills and adaptive behavior because the child may need to adapt emotions or behavior to the setting in order to successfully use the learned skills. Riggio (1986) described social skills as basic skill components that should be used in combination to develop more complex social patterns in interactions. This suggests that the acquisition of basic social skills is not synonymous with socially competent behavior and the ability to use these skills to form and maintain relationships in generalized settings. Spence (2002) refers to social competence as the application of the learned social skills to develop and maintain friendships and relationships. Gutstein and Whitney (2002) defined social competence more explicitly as "the skills and strategies that allow individuals to have meaningful friendships; forge close, emotion-based relationships; productively collaborate with groups, teams, and work partners; manage public social settings; and participate in family functioning" (p. 161). All of the research referenced ! %! suggests that the acquisition of social skills in not adequate to produce pr-social behavior in individuals, but there must also be integration of these skills in generalized settings and interactions in order for the child to be considered socially competent and to benefit from the social skills training. The teaching of social skills, and more importantly, social competence is necessary, but difficult to do effectively. In order to become more effective in the efforts to teach children how to be socially competent, there is a need for more research to identify what makes social skills training programs effective. Common Components of Social Skills Programs In order to effectively teach social skills to children, multiple components are often combined within training programs. The goal is to teach the basic social skills to ensure children have the ability to perform the skills and then train them to achieve social competence in order to effectively use these skills in multiple social situations. Table 1 provides an overview of the training sequence of some common social skills programs. There are also many programs that offer group facilitators a guide to develop their own lessons or activities that can be used to create lessons, but do not have a set lesson structure that is provided in the manual. Many of the social skills programs that are currently used in schools and clinical settings share commonalities in their focus and their method of instruction. McConnell (2002) divided the current social skills programs into five categories. The first category is environmental modification strategies. The focus of these types of interventions is on making changes to the environment in order to encourage social interactions. The second ! &! Table 1 Training Sequence of Common Social Skills Programs Social Skills Programs The Tough Kid Social Skills Book (Sheridan, 1995) Review Introduce new skill Student pole plays Group discussion/Feedback Set goals/Behavior contract ACCEPTS Definition/Guided discussion (Walker, McConnell, Holmes, Todis, Positive example Walker, & Golden, 1983) Negative example Review/Restate definition Positive video example Activities Criterion role-plays Informal contracting Second Step Program (Committee for Children, 1998) Story/Discussion Role-plays Wrap up Homework ! '! category of interventions is collateral skills interventions that teach skills, such as play and language, to improve social interaction. Another type of intervention is peer-mediated interventions that use trained typically developing peers to teach skills and encourage social interaction. Child-specific intervention is another type of intervention that teaches specific social skills to children for them to use in their social interactions. The last type of intervention described is comprehensive interventions that combine two or more types of the interventions previously discussed. Many programs use a demonstration-prompt-practice model to teach social skills to children (Morgan & Jenson, 1988). The lesson format for a demonstration-prompt-practice model, in its most basic approach, begins by teaching the basic skills to children and modeling how to use the skills appropriately, then the children are prompted to use the skill and situations that would be appropriate for using the skills are identified, and lastly, the children practice, or rehearse, by using the skill in multiple scenarios. It can also be helpful to incorporate other components into the training sequence. When teaching the concept of a new skill, it is helpful to provide a rationale for why it is important to use the skill, teach the steps of the skill explicitly, and then model examples and nonexamples of how to use the skill appropriately. This provides a good foundation for learning a new skill. Also, it is important for the children to practice only the appropriate use of the skill. As the children practice the skill, the facilitator should provide feedback to increase performance during various role-play situations. Multiple opportunities to practice with varying scenarios provides a wide array of applications of the skill. Practice can continue until the children are able to perform the skill to criterion. ! (! Inclusion of a behavior management system into the lessons can help decrease any problem or interfering behavior. The use of group contingencies can be an effective way to increase compliance during the training process. It is also helpful to integrate a way to reward or reinforce the children's appropriate use of the skills. Homework is another common component of social skills training that can help to increase the use of the skills outside of the training setting. It also provides a way to review the skill at the beginning of the next training session. Worksheets are often used as a homework component, but a self-monitoring system to help children track their use of the skill outside of the sessions can also be an effective way to use homework. Many programs vary in the types of skills that are taught. Assessment of skill deficits and individual needs of the child can help to identify the most appropriate programs or individual lessons that may be most beneficial to the children. Many programs target specific skills in the area of foundational skills, compliance skills, friendship making skills, cooperation skills, bullying skills, and coping skills. Table 2 provides a list of the skills and skill areas that some common social skills programs address in their training program. As more research has been conducted in the area of social skills training, more effective methods have been identified. Lane, Bocian, MacMillan, and Gresham (2004) outline effective strategies for implementing interventions in schools, which include (a) identifying students for participation, (b) identifying specific skill deficits and designing the intervention program, (c) organizing intervention groups, (d) preparing intervention leaders, (e) implementing the intervention, and (f) monitoring student progress. This is consistent with the process that should be, and often is, utilized when servicing students ! )! Table 2 Skills Taught in Common Social Skills Programs ACCEPTS The Tough Kid Social ASSET (Walker, McConnell, Skills Book (Hazel, Schumaker, Holmes, Todis, Walker, (Sheridan, 1995) Sherman, & Sheldon- & Golden, 1983) Wildgen, 1981) CLASSROOM SKILLS SOCIAL ENTRY Giving positive feedback Listening to the teacher Body basics Giving negative feedback Doing your best work Joining in Accepting negative feedback Following classroom rules Recognize feelings Resisting peer pressure BASIC INTERACTION Express feelings Problem-solving Eye contact INTERACTIONS Negotiations Using the right voice Have a conversation Following instructions Starting Play cooperatively Conversation Listening PROBLEM SOLVING Answering Solving problems Making sense Using self-control Taking turns talking Solving arguments ` Questioning Dealing with teasing Continuing Dealing with being left out GETTING ALONG Accepting "no" Using polite words Sharing Following the rules Assisting others Touching the right way MAKING FRIENDS Grooming Smiling Complimenting Expressing anger Making friends COPING When someone says "no" When someone teases you/tries to hurt you When things don't go right ! *+! with individualized special education services. They also posit that treatment integrity should be an essential component in this process, but it is often not included in school-based interventions. This is essential when using a manualized and researched program, to ensure that the same methods that were used during the research, which yielded particular results, are also present in the field to increase the likelihood of obtaining similar results to the research studies. Cook, Landrum, Tankersley, and Kauffman (2003) identified the ability to bridge the gap between research and practice as a setback in the implementation of social skills programs and suggested a stronger focus on treatment fidelity as a possible solution to this problem. Wang and Spillane (2009) agree that if the research is finding only questionable results in a controlled setting when the intervention is delivered by researchers, it is unrealistic to expect noteworthy results in an applied setting when delivered by teachers and parents. There are many manualized social skills programs available, some specific to populations and others more general to be used with multiple populations. Regardless of the type of social skills program, Gresham (1995) recommends some fundamental elements to be included in social skills programs: identify skills that need to be remediated, teach and model the skills, target skills to be taught, coach and prompt proper use and application of the skills, provide opportunities for the skills to be rehearsed, provide reinforcement and feedback for the skill use, implement reductive procedures, and facilitate generalization. Overall, many of the manualized social skills curriculums utilize a similar structure for their lessons, but with slight variations in the methods used. This common ! **! structure can be used and modified depending on the needs of the children, but it may also be important to include more strategies that are identified in the research as effective. Current Social Skills Programs Social skills training has been a research focus for many years and targeted toward many varying populations due to the social deficits that are found across disorders. Overall, programs are aimed at teaching and practicing pro-social behaviors with the intent of overcoming the skill or acquisition deficit, but few have been able to overcome the performance deficits associated with the use of social skills. The majority of current social skills programs include some or all of the components previously identified as effective, but they are applied differently. Similarly, all of the programs differ in the amount of research supporting the use of the program and the amount of research-based practices that are incorporated into the program. A sample of some of the common social skills curriculum is listed in Table 3. All of the programs listed are widely used by practitioners and many of them have research supporting their programs. Many of the components that have been found to be effective in these programs have been incorporated into the Superheroes Social Skills program, which is the focus of this study. Gray (1994) has published multiple books about social stories, which are now widely used in social skills curriculum. Social stories are developed and used by writing a story that incorporates use of the target skill in a specific situation. The child then learns how to use the skill by reading the story or having it read to them. Social stories are incorporated in the Superheroes Social Skills program in the form of comic books ! *"! Table 3 Common Social Skills Programs Common Programs Social Competence Intervention Program Guli, Wilkinson, & Semrud-Clikeman, 2008 Skill Streaming Goldstein & McGinnis, 1984 Prepare Curriculum Goldstein, 1988 Navigating the Social World McKinnon & Krempa, 2005 Building Social Relationships Bellini, 2006 The New Social Story Book Gray, 1994 Superflex: A Superhero Social Madrigal & Winner, 2008 Thinking Curriculum Package Think Social Winner, 2006 Cool Kids Fister-Mulkey, Conrad, & Kemp, 1998 The ACCEPTS Program Walker, McConnell, Holmes, Todis, Walker, & Golden, 1983 ASSET Hazel, Schumaker, Sherman, & Sheldon-Wildgen, 1981 ! *#! featuring the characters from the curriculum. Madrigal and Winner (2008) have developed a social skills curriculum that incorporates a "superhero" theme, but with a focus on social thinking. They utilize specific characters that have to learn skills to overcome a certain social deficit. In contrast, the Superheroes program in this study uses two superheroes and a robot sidekick to teach the specific social skills to children. Bellini, Akullian, and Hopf (2007) found both self-modeling and video-modeling to be an effective means to teach social skills to children. By using peer video-modeling, children are able to learn the social skills better than if an adult teaches the lessons in a didactic format. This program also uses video-modeling with peers who are shown using the skill being taught in multiple situations. All of these components and others were incorporated into the Superheroes Social Skills program as a way to produce an effective curriculum for children with ASD. Due to the large number of research-based methods incorporated into this program, it may prove effective for other populations, such as those with high incidence disabilities who also exhibit social deficits. Current Social Skills Programs for Children with Externalizing Behaviors There are also many programs that have been developed for children with externalizing behaviors. Effective social skills training is essential for this population due to the severe detrimental effects their behavior has on their ability to develop and maintain social relationships with peers and adults. Table 4 provides a sampling of some of the commonly used social skills programs specifically for children who exhibit externalizing behaviors. ! *$! Table 4 Common Social Skills Programs for Externalizing Behaviors Current Social Skills Programs Incredible Years Webster-Stratton, 1984 Aggression Replacement Training Goldstein & Glick, 1986 Tough Kid Social Skills Book Sheridan, 1995 First Steps to Success Walker et al., 1997 Second Step Program Committee for Children, 1998 Steps to Respect Committee for Children, 1998 Olweus Bullying Prevention Program Olweus et al., 2007 The Incredible Years Program (Webster-Stratton, 1984) includes parent, teacher, and child training programs. The main component of these programs is video vignettes that are viewed by the small group and then analyzed through a group discussion that is led by the facilitator. The Dinosaur School, which is the child component of the program, also uses life-size dinosaur puppets to help teach appropriate skills. There have been many research studies completed on the effectiveness of the Incredible Years Program, both by the developer and independent evaluators. The majority of studies (Taylor, Schmidt, Pepler, & Hodgins, 1998; Webster-Stratton & Hammond, 1997; ! *%! Webster-Stratton, Reid, & Hammond, 2004) found moderate to large effect sizes for the use of the parent, teacher, or child programs when studied individually and when combined. Aggression Replacement Training (Goldstein, Glick, Reiner, Zimmerman, & Coultry, 1986) is a program that was developed to provide children with anger control training and pro-social behaviors through skill streaming. Studies by the developers indicate that this program decreased aggressive behaviors and increased positive social behaviors (Goldstein & Glick, 1994). Nugent, Bruley, and Allen (1999) found that Aggression Replacement Training could be an effective component of a multicomponent intervention to increase pro-social behaviors and minimize aggressive behaviors. The Tough Kid Social Skills Book (Sheridan, 1995) focuses specifically on teaching children with externalizing behaviors pro-social skills. The main teaching strategies employed in these lessons are the discussion introducing the skill to be learned and role-playing activities for the children to practice the skill. One study by Fenstermacher, Olympia, and Sheridan (2006) used the content of this program and delivered in a computer format. They found that the children with ADHD improved their social problem-solving skills. First Steps to Success (Walker et al., 1997) is a program developed as an early intervention program to prevent anti-social behaviors in children. In response to their exposure to risk factors, many children develop anti-social behavior that can escalate as the children get older. This program identifies risk factors present for these children and then provides school intervention and parent training. Results of studies by the developers and independent evaluators (Loman, Rodriguez, & Horner, 2010; Sprague & ! *&! Perkins, 2009; Walker et al., 1998) indicate robust results for the effectiveness of this program when used with children with antisocial behaviors. Project Second Step (Committee for Children, 1998) is another program designed to increase social skills and decrease antisocial behavior in youth. Taub (2001) found that the program was effective for children in a rural school. Grossman et al. (1997) found similar results for elementary students. Another program developed by the Committee for Children is the Steps to Respect (1998) program that combines friendship-making skills and anti-bullying into one curriculum. Frey et al. (2005) did find a reduction in aggressive and argumentative behavior on the playground following implementation of the program. This was determined by the use of unbiased observations during recess. The Olweus Bullying Prevention Program (Olweus et al., 2007) was adapted from a program first developed in Norway. It is available for use as a classroomwide program or as a schoolwide program. One study by Bauer, Lozano, and Rivara (2007) found that there was no overall positive effect in a middle school sample. This is consistent with other research in the area of bully prevention. Overall, there are programs intended for use with children with externalizing behaviors that show some effectiveness in the research. There is a need to continue research to determine what factors make these programs effective for this specific population and what factors could increase the effectiveness and make the results more long-lasting. ! *'! Evidence-Based Practice Due to the wide range of detrimental effects that social skills deficits can have on children, there has been a large focus from researchers on developing social skills interventions that are effective. Social skills programs are widely used in schools and clinical programs for children with social deficits as an attempt to improve their levels of functioning and ability to build and maintain peer relationships. It is necessary to further study and develop programs that are effective in improving the skills necessary for individuals to thrive socially, academically, and in society. The National Association of School Psychologists (NASP) supports the need for use of evidence-based practice (EBP) by school psychologists. Hoagwood and Johnson (2003) define evidence-based practice as "a body of scientific knowledge, defined usually by reference to research methods or designs, about a range of service practices" (p. 4). Cournoyer and Powers (2002) recommend that the way school psychologists make decisions and provide services be based on the use of evidence-based practices. This means that practitioners use services that have research indicating that the intervention is likely to be beneficial to the person you are using it for and that the practitioner will measure the effects of the intervention on the individual throughout treatment. By doing this, the intervention is likely to produce predictable effects that are beneficial for the individual. Kratochwill and Shernoff (2003) identified five elements that are needed to effectively utilize evidence-based practice. The first is that there is collaboration between researchers, trainers, and practitioners to ensure that the interventions being developed are effective in practice environments. The second need is for practitioners to use ! *(! manualized treatments to increase the treatment fidelity and the likelihood of efficacy when transferring intervention implementation from a research setting into practice. Along with the need for practitioners to use a manual for implementation, it is suggested that more specific practical guidelines be provided to make treatments even more effective. The fourth consideration is the need for professional development for graduate students, trainers, and practitioners to help them make better applications of the interventions to specific practice settings. Finally, it is recommended that a scientist-practitioner model is most effective in supporting the development and research of interventions in practical settings. The American Psychological Association (APA) also provides guidelines for the development, evaluation, and use of evidence-based practice. APA's Presidential Task Force on Evidence-Based Practice (2006) defined evidence-based practice as "the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences" (p. 273). This definition is also very similar to the definition of evidence-based practice as defined by the Institute of Medicine (2001). The American Psychiatric Association also developed similar guidelines to help physicians with decision-making about the best form of treatment for patients. There is consistency between the definitions and this common language may foster a higher level of integration between the medical and mental health communities. Also, this definition and the guidelines set into place by APA identifies specific goals to make mental health services more cost effective and to make practitioners more accountable for their actions and treatments. Similarly to NASP, APA recognizes that collaboration between ! *)! researchers and practitioners is essential to developing and implementing evidence-based practices. APA Division 12 (Clinical Psychology) and Chambless et al. (1998) have developed criteria for evidence-based practice by delineating between well-established treatments and probably efficacious treatments. Well-established treatments can be determined as such in one of two ways. The first requires at least two good between-group design experiments demonstrating efficacy in that it is more effective than a psychological placebo, other treatment, or an already well-established treatment. The second way in which a well-established treatment can be identified is through a large series of single-case design experiments, with 9 or more participants, which demonstrate efficacy. These single-case design experiments must have used good experimental designs and provide evidence of change by comparing the intervention to another treatment. Both of these methods to identify well-established treatments require treatment manuals, clearly specified client sample characteristics, and effects demonstrated by at least two investigators. Probably efficacious treatments require two experiments that indicate the treatment is superior to a waiting-list control group or a small series of single-case design experiments with three or more participants and otherwise meeting the criteria for a well-established treatment. The guidelines set up by Division 12 give very specific criteria for practitioners to determine the efficacy of interventions for their patients and researchers to develop well-established interventions. While APA Division 12 and Chambless et al. (1998) provided very clear guidelines for the classification of evidence-based practice, O'Donahue and Ferguson (2006) have identified some weaknesses in this system of classification. In the criteria ! "+! previously defined, the determination of evidence-based practice is based on statistical significance rather than clinical significance. Statistical significance is determined by how much chance affects a difference in the results, but clinical significance refers to how meaningful the change is to the client. The second weakness argument against this determination of evidence-based practice is that the decisions are based on efficacy (the treatment is beneficial for patients) rather than effectiveness (if an efficacious treatment will not only be effective in research settings, but also in community settings/private practice). The third weakness found in the EBP criteria is concerned with the issue of heterogeneity versus homogeneity. Most studies that meet this criterion exclude subjects who present with comorbid diagnoses, although in community and private practice settings, the patients are often presenting with multiple conditions. The fourth and last weakness identified by O'Donahue and Ferguson recognizes the bias against inclusion of single-subject, withdrawal, and multiple-baseline research designs due to the requirement of inferential statistics and comparisons to a control group. Many groups specific to education have also provided information in regards to evidence-based practice. The Council for Exceptional Children (CEC; Odom et al., 2005) has helped by identifying some criteria specifically intended to assist teachers in the identification of research-based practice. Currently, there is not very extensive research on practices that meet EBP criteria and are also relevant for use in an educational setting. The Department of Education has attempted to provide more educationally relevant criteria to determine if practices are evidence-based. The Institute of Educational Sciences (IES) (2003) has recognized criteria for interventions to be ! "*! considered as having "strong" evidence or "possible" evidence of effectiveness. In order to meet the criteria for having "strong" evidence, an intervention must have been effective in well-designed and implemented randomized controlled trials in two or more typical school settings. "Probable" evidence of effectiveness is found in studies with randomized controlled trials, but may not be able to meet the stringent requirements for having "strong" evidence. The IES places a great deal of importance on randomized controlled trials as a research method. Aspects of the randomized controlled trials that the IES also discusses with strong emphasis are accurate outcome measures, long-term outcomes, detailed description of treatment groups, indication that the intervention groups are systematically equal, and that the results are statistically significant. These criteria should all be met before an educational intervention is considered to have "strong" research evidence of effectiveness. The American Speech-Language Hearing Association (ASHA, 2004) has developed criteria similar to those developed by APA Division 12 as a way of determining the level and amount of research supporting an intervention. They categorize interventions into levels ranging from Level I to Level IV. Level I includes interventions that have been studied through a meta-analysis with one or more studies having randomized designs, Level II can include controlled studies and quasi-experiments, Level III includes case studies and other nonexperimental designs, and Level IV would include interventions without research, but with expert support. Similar to the guidelines developed by the APA, these guidelines give a clear division between the necessary level of research support for interventions and a means to gauge the likely ! ""! degree of expected outcomes. This is very helpful to educators as they attempt to make decisions about how to best serve their students. The National Autism Center (NAC, 2009) has defined three categories of interventions: Established, Unestablished, and Emerging. Some of the criteria for Established treatments include having research providing evidence of beneficial effects, the expectation of long-term beneficial effects, and evidence that the treatment does not produce harmful effects. While these treatments have been shown to be effective, they should not be expected to be effective for every child and multiple types of treatments may need to be tried before the most effective treatment for that individual is found. Unestablished treatments differ from Established treatments in that there is little or no evidence to support them. Unestablished treatments also may have been found to produce little positive effect or negative effects in the research. Emerging treatments are those that do not have enough research to support effectiveness or lack of effect as an intervention. These should be used with caution, as they do not have enough empirical support to determine how effective or detrimental they may be. These guidelines require the practitioner to take a lot of responsibility and use good professional judgment when choosing treatments for patients. All of these agencies provide guidelines for determining good practice and best practice to be used by professionals in the schools. Superheroes Social Skills has many of the components that have met these stringent guidelines incorporated into the manualized program. ! "#! Meta-Analyses Meta-analyses are used as a means to combine the results of several studies in order to better determine the degree of effectiveness of similar interventions. According to Glass (1976), they are used as a "statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings" (p. 3). Meta-analyses are used because it can be very difficult to detect statistically significant results from individual studies. Oftentimes, there are not enough participants in individual studies to provide the statistical power needed to show large effects. As suggested by Collins et al. (1992), in order to prove that a drug is designed to reduce the risk of disease by 10%, a sample size of 10,000 would be needed in each of the treatment groups to detect any effect with 0% accuracy. By combining individual studies into a meta-analysis, the ability to detect statistical significance is increased. Blimling (1988) identified four main purposes of meta-analyses: to describe existing studies of a treatment, to determine overall effectiveness of the treatment, to determine influence in the outcome of the treatment, and to quantify the outcome in terms of magnitude and significance. Davis and Crombie (2001) also pointed out some advantages of using meta-analytic research. Using meta-analyses allows people to see the average effects from multiple applications of similar interventions by producing a larger number of participants than the individual studies. This process typically reflects more accurate effects because of the larger sample being used. Another benefit is that meta-analyses are typically more objective than traditional studies and reviews that can often be biased by the researcher or reviewer. ! "$! In order to maintain the integrity of the results obtained from meta-analyses, it is essential to follow the process that is defined for conducting a meta-analysis. The process begins when the researcher develops a question and defines inclusion criteria for the studies that will be used. By developing the inclusion criteria at the beginning, the researcher is unable to later exclude studies based on personal preference, thus increasing the objectivity of the studies used. The studies that are chose should have methodological soundness and enough data provided to compare between the studies. Meta-analyses are an objective and highly effective way to evaluate the efficacy of interventions. They provide more accurate information based on the results of multiple research studies and they provide more guidance toward possible areas for future research. Efficacy of Social Skills Programs Social skills are an important part of development for children and there are many populations that do not naturally acquire them, such as children with Autism Spectrum Disorder (ASD), children with internalizing disorders (e.g., depression and anxiety), and children with externalizing disorders (e.g., conduct disorder and ADHD). Programs targeted at helping children develop functional social skills have been developed and targeted for use with many specific groups, including children with depression, behavior disorder, anxiety, and ASD, although the research does not always indicate positive or even neutral results for social skills training (Arnold & Hughes, 1998; DuPaul & Eckert, 1994). ! "%! Many social skills programs have been developed, but in current research, they have not been found to be effective in increasing social skills or helpful in generalizing skills across settings (Arnold & Hughes, 1998; DuPaul & Eckert, 1994). It is concerning that despite the limited benefits of social skills training, these programs are widely used in school settings (Landrum, Tankersley, & Kauffman, 2003). There are also many meta-analyses that have been conducted to determine the effect size of social skills training on children. Many studies have found the effect size to be small according to Cohen's (1988) measurement for effect sizes, which indicates that below 0.20 is a small effect size. Forness and Kavale (1996) conducted a meta-analysis on social skills programs for children with learning disabilities and found small effects. The meta-analysis included 83 independent studies done to determine the effect of social skills training conducted specifically for children with learning disabilities. The average effect size for all of these studies was 0.21. In a later review of this and other meta-analyses, Forness (2001) found that another meta-analysis (Quinn, et al., 1999) also reported a small average effect size of 0.20 for the 35 studies that had met inclusion criteria. Similarly, another meta-analysis by Forness and Kavale (1999) concluded an effect size of 0.21 for students with specific learning disabilities who received 3 hours per week of social skills instruction. This is another study with a small effect size found for the use of social skills training as an intervention. One conclusion drawn from this research was that students classified as having a disability and requiring special education services are more resistant to social skills interventions than other children who are exhibiting social skills deficits. ! "&! Lloyd, Forness, and Kavale (1998) concluded that social skills training programs, in general, are not very effective. Some promising techniques suggested for increasing the effectiveness of social skills included intervene early, monitor students' progress and provide positive consequences for improvement, teach cognitive behavioral self-management, teach academic and cognitive skills directly and systematically, use behavioral techniques to promote the acquisition of academic and social behaviors, and teach mnemonic strategies for understanding and remembering what the students learn. Peer tutoring as also identified as a strategy that may be effective. The meta-analysis by Bellini, Peters, Benner, and Hopf (2007) focused on school-based social skills programs for children with ASD. Their study measured the effects of social skills training on children's group play, social initiations, and social responses. The results indicated that the interventions implemented in the schools produced moderate maintenance effects and low generalization effects of group play, social initiations, and responding behaviors for the participants. Bellini also found that the social skills training was less effective when taught outside of the natural setting. Bellini and Akullian (2007) conducted a study that focused on the effects of video-modeling and video self-modeling when used to teach social skills to children with ASD. Their meta-analysis included studies that measured the effect of modeling training on social-communication skills, functional skills, and behavioral functioning. Overall, the studies produced moderate effects for the three variables, with the Percentage of Non- Overlapping Data Points (PND) being 80%. Specifically for the effects in the dependent variables, functional skills had the highest PND of 89%, social-communication skills had a PND of 77%, and behavioral functioning had a PND of 76%. Unlike previous research, ! "'! maintenance effects had a PND of 83% and generalization effects had a PND of 74%, indicating moderate effects in these areas. Little difference was found between the effects of video-modeling and video self-modeling. Wang and Spillane (2009) also studied the effects of social skills training with children with ASD and found them to be generally ineffective. Social stories, peer mediation, and video-modeling were identified from previous research as being evidence-based practices, but the current study concluded that PND resulted in video-modeling as the only evidence-based practice. Maag (2006) conducted a review of previous research reviews and concluded, "The state of social skills training with students with emotional and behavioral disorders seems to range from dismal to guarded optimism" (p.14). Some of he concerns that were identified I current social skills training programs included a lack of generalization, not focusing on socially valid behaviors, the training was not tailored to the individual students' deficits, lack of socially valid outcome measures, and lack of treatment fidelity. These results may indicate deficiencies in current research, but also provide some insight into what should be incorporated into new programs to increase effectiveness. Zhang (2008) conducted research on the effects of using peers to mediate social skills interventions for children with ASD. Peer mediation as an intervention was found to have a large effect size of 1.46, follow-up results also had a large effect size of 1.49, as did generalization with an effect size of 1.51. Miller (2006) also found peer mediation to be an effective form of social skills intervention. This meta-analysis found peer mediation had a large effect size of 3.27, as did collateral skill interventions (ES=2.37) and child-specific interventions (ES=2.19). Both of these studies provide support for use ! "(! of peer-mediated interventions as an evidence-based practice when used to teach social skills to children with ASD. A meta-analysis by Lee, Simpson, and Shogren (2007) evaluated the effectiveness of many self-management techniques. The techniques included in this study were self-monitoring, self-assessment, self-evaluation, self-observation, self-recording, self-instruction, and self-reinforcement. Results combined for all forms of self-management produced a PND of 81.9%. The results of this study imply that self-management may be an effective intervention for children with ASD. Video-modeling, self-management, peer mediation, and social stories are found in the research as being helpful to some children in learning, generalizing, and maintaining social skills (Bellini & Akullian, 2007; Bellini, Akullian, & Hopf, 2007; Charlop-Christy & Danshevar, 2003; Hagiwara & Myles, 1999; Miller, 2006; Thiemann & Goldstein, 2001). The use of these components can be effective for children learning social skills in a group setting. While there is not a complete consensus in the literature that these interventions are helpful to all children, it is encouraging that they are effective for some when used alone and could be even more effective when combined with other evidence-based approaches. Schneider (1992) is one of the few meta-analyses to indicate moderate or large effect size for social skills training. This study found an overall effect size of 0.87, which is considerably larger than most of the other studies. This was also one of the few meta-analyses that found a moderate effect size at follow-up (ES=0.75). Schneider did find that children who were categorized as aggressive showed smaller effects than the children who were described as withdrawn, implying that social skills training may be more ! ")! effective for children with internalizing behaviors rather than those with externalizing behaviors. A meta-analysis by Quinn et al. (1999) found an effect size of only 0.20 for children with emotional and behavior disorders whom received social skills training for an average of 2.5 hours per week for 12 weeks. This is disconcerting when the reality of the maximum amount of time social skills training occurs for students in the schools in one hour per week. The results indicate that only about 58% of the students who received the treatment showed improvement in their pro-social behaviors. One study aimed at determining the effectiveness of social skills training for children with conduct disorders not only failed to show benefits from group social skills training, but actually showed evidence of detrimental effects due to the effects of grouping externalizing children (Arnold & Hughes, 1998). In social skills groups for children with behavior disorders, it has been found that the undesirable behaviors were actually enhanced due to the encouragement and experience sharing between the individuals in the group. Thus, social skills interventions for children with behavior disorders are not deemed effective unless there are neutral peers without behavior disorders in the group as well. Use of peers to mediate instruction is an embedded component in the Superheroes Social Skills program. However, another study by Beelman, Pfingsten, and Losel (1994) found that social skills programs taught with children who have externalizing disorders had an initial moderate effect size of 0.48, but these effects were not maintained over time. This would suggest that while the treatment produced a higher effect size as measured during the ! #+! treatment period, the long-term effects of the social skills training were not beneficial for the participants. Quinn, Kavale, Mathur, Rutherford, and Forness (1999) found the effect size of social skills instruction for children with emotional and behavior disorders to be 0.199. However, the same study did find higher effect sizes for children with anxiety, suggesting that social skills training may be more effective for some children with internalizing disorders. The moderate effect size of 0.422 for social skills training for children with anxiety in this meta-analysis was based on eight individual studies. A more recent meta-analysis conducted by Spence, Donovan, and Brechman-Toussaint (2000) also provides support for use of social skills for children with anxiety. This study paired social skills training with cognitive-behavioral therapy to reduce school-related anxiety. The results of this study indicate that this treatment was effective and was able to be maintained at 12 months after the completion of treatment. This research also provides evidence that social skills training might be effective for children with internalizing disorders. Miller, Lane, and Wehby (2005) used social skills training in a self-contained classroom with children with high-incidence disabilities. Their study found reductions in inappropriate classroom behavior, and increases in time engaged in academic tasks. Unfortunately, these effects were not reflected in the classroom behavior management system and the teacher and student ratings of social validity were low, which may indicate an effective treatment that lacks generalization. Maag (2005) reviewed studies that provided social skills to children with emotional and behavior disorders and found many areas for improvement in this area in the future research. This study concluded that the target behaviors were not socially valid ! #*! and did not match the performance deficits the children had. There was also a lack of generalization and treatment fidelity, as well as, minimal changes in peer acceptance being observed. It would be important to address some of these issues by assessing the needs of the students, training with peers, and incorporate generalization strategies. Gresham, Cook, Crews, and Kern (2004) also studied social skills training for children with emotional and behavioral disorders. Some of the conclusions drawn included that social skills training is an effective intervention for children with emotional behavioral disorders. Specifically, it can be effective for children with aggression and externalizing behaviors, children with internalizing behaviors, and children with antisocial behavior patterns. Ineffective or inaccurate outcome measures was one of the weaknesses found in previous studies, implying that future research in this area should consistently use better outcome measures. Interventions specific to bullying have also been studied more frequently in the recent past. This is an important area of research to consider when working with children with externalizing behaviors who are displaying aggressive behavior toward others. There are a few meta-analyses that have been conducted on the research regarding bullying prevention programs and indicate small effect sizes for the programs currently being used (Livingston, 2008; Merrell, Gueldner, Ross, & Isava, 2008; Wong, 2009). These studies did indicate that there was a higher reduction in victimization as a result of the intervention, but less of an effect on the reduction of bullying behaviors. It is unclear from these studies what factors of the programs are most and least effective, as well as why there seems to be a larger effect on the victim behavior than bullying behavior. ! #"! The current research on the use of general social skills programs for children with various disabilities does not seem to provide a large amount of evidence for their effectiveness. While some of the groups showed positive effects, many did not. There are many programs that may be effective for various groups of children, but have not yet been the focus of research at this point. This is an area in great need of more research to support decisions being made for programs developed for and conducted with children. Some research has indicated that existing social skills programs are not effective and the effects may not be generalizable to multiple settings. DuPaul and Eckert (1994) found that many social skills programs were ineffective because the skills being taught were not generalizable into natural situations where the use of the skill would be beneficial to the child. They used the term "train and hope" method of teaching social skills to refer to the common practice of teaching skills to children and then hoping that the children use the skills in generalized settings, but without any further training to encourage the skills use. DuPaul and Eckert also found that performance deficits impeded the generalization of the skills because knowledge of the skills was acquirable, but the self-control and impulsivity levels of the children kept them from being able to utilize the skills appropriately in actual situations. This would imply that social skills would need to be taught to children who were able to overcome the performance deficits that might be present in order for there to be positive and generalizable effects. Generalization of Social Skills Morgan and Jenson (1988) define generalization as having occurred "when the learner exhibits the target behavior outside the training setting, with no specific ! ##! intervention" (p.156). There are four types of generalization identified by Drabman, Hammer, and Rosenbaum (1979): generalization across time (maintenance), generalization across setting (stimulus generalization), generalization across behaviors (response generalization), and generalization across subjects. All four types of generalization are relevant when teaching social skills to students in a school setting. It is desirable that the social skills are used in multiple settings where intervention is not occurring, they continue to be used after the training has ended, and the skills are applied to multiple and varied situations. Sheridan (personal communication, 2001) expands on previous definitions of generalization to be "the use of appropriate social skills in a variety of settings and with a variety of people that were not part of the training situation (such as in classrooms, playgrounds, and hallways; with teachers, classmates, and others)" (slide 2). The desire is to teach skills that will be used in all possible settings and with all of the people the child interacts with, not just in the training setting or in the classroom. Unfortunately, generalization of social skills is often not achieved from current social skills training programs. Gresham provides a framework for the difference between learning social skills and being able to generalize the skills. In the matrix, Gresham delineates between the difference of learning a skill, or acquiring it, and performing the skill in a generalized setting. For many children, they are able to learn the skill, but are not able to perform the skill outside of the training setting, resulting in a generalization deficit. There are two types of performance deficits and two types of acquisition deficits. Social performance deficits occur when an emotional arousal response is absent and self- ! #$! control performance deficits occur when the emotional arousal response is present. For acquisition deficits, social skill deficits occur when the emotional arousal response is absent and self-control deficits occur when the emotional arousal response is present. These deficits are dependent on the ability to learn the skill and the child's ability to adapt the skill to various situations based on their emotional arousal. These factors should be considered when aiming to increase generalization of learned skills. DuPaul and Eckert (1994) termed the problem with generalization of social skills as "now you see them, now you don't" (p. 113) because the effects of social skills training may be seen initially, but are not present after training has been completed. In their review of the literature, they concluded that some environmental programming and training in the "real world" setting helped to increase generalization and maintenance. Overall, the effects of social skills training are not present beyond the training setting. This implies that different strategies would need to be employed that focus on generalization and maintenance if there is to be any practical application and result of social skills training with children. Children need to learn the skills and their specific steps, but also learn to control their emotional arousal and be able to adapt the skills in generalized settings when situations occur that would require them to use the learned skills. Enhancing Generalization While generalization has been identified as a necessary component of social skills training (DuPaul & Eckert, 1994), strategies to increase generalization are often not embedded components in many social skills programs. Stokes and Baer (1977) refer to ! #%! the commonly used methods for promoting generalization of social skills as the "train and hope" method. This is often seen when students receive pull-out social skills training and then school staff hope they use the skills in generalized settings. Oftentimes, the skills are not used outside of the training setting. Many strategies to increase generalization have been identified by Morgan and Jenson (1988). Some of the strategies they suggest include sequential modification, natural contingencies of reinforcement, multiple teaching examples, training loosely, indiscriminable contingencies, common stimuli, and self-management. Sequential modification requires that the same intervention procedures be applied in all settings that the child is exposed to, which can be difficult to implement with multiple teachers in multiple classrooms. Natural contingencies of reinforcement is a concept that would imply that increased social skills are naturally reinforced by teachers and peers by positive response to the child's use of desired behaviors. This is often not the case because positive behaviors are often ignored rather than reinforced. Another suggested practice is providing numerous teaching examples for the behavior to be learned. If the student learns the skill as it applies in multiple situations, there is a higher likelihood that the student will use the skill in similar situations outside of the training setting. If the student practices the skills in multiple situations, they may be more easily adapted to similar situations in the natural setting. Training loosely is a strategy that Baer (1981) suggests can increase generalization through variation of reinforcement, classroom environment, teachers, and teaching techniques. Indiscriminable contingencies can be used to vary the reinforcement and behavior that is being reinforced so it becomes unpredictable for the students. Stokes and Baer ! #&! (1977) suggest that making the training setting and the generalization setting more alike will help the skills taught in the training setting to be used more in the generalization settings. The last strategy that is suggested to increase generalization is teaching self-management procedures. This helps students learn how to monitor their own behavior in generalized settings leading to increased use of the skills in these settings. Sheridan (2001) offers similar strategies for increasing generalization. She suggests teaching meaningful skills to the student and adults that are most important based on an assessment of the child's environment. She also suggests bringing the "real world" into training and bringing training into the "real world". This is done by making the training environment similar to the natural environment and by conducting training in the natural environment. Sheridan encourages classwide and schoolwide programs to teach all kids by providing situations in which the skills can occur, using peers as mediators, including all adults (teachers, administrators, aides, parents, peers, etc.), train loosely, using reinforcement, and using cueing techniques in order to increase generalized use of the skills being taught. Reeve et al. (2007) aimed to increase the helping behavior of children with ASD. During the training trials, the researchers used modeling, video-modeling, and reinforcement to encourage the children to acquire the desired skills. Results of this study indicated that the helping behavior generalized outside of the training setting and was maintained at a 60-day follow-up. Conclusions may be drawn that modeling, video-modeling, and reinforcement can be effective strategies for increasing generalization of some learned skills. ! #'! Stokes and Baer (1977) conclude in their article that there is no such thing as generalization without programming. There must be techniques incorporated into programs to foster generalization of skills learned. The tactics they recommend using to increase generalization include: 1. Look for a response that enters a natural community; in particular, teach subjects to cue their potential natural communities to reinforce their desirable behaviors. 2. Keep training more exemplars; in particular, diversify them. 3. Loosen experimental control over the stimuli and responses involved in training; in particular, train different examples concurrently, and vary instructions, social reinforcers, and backup reinforcers. 4. Make unclear the limits of training contingencies; in particular, conceal, when possible, the point at which those contingencies stop operating, possible by delayed reinforcement. 5. Use stimuli that are likely to be found in generalization settings in training settings as well; in particular, use peers as tutors. 6. Reinforce accurate self-reporters of desirable behavior; apply self-recording and self-reinforcement techniques whenever possible. 7. When generalizations occur, reinforce at least some of them at least sometimes, as if "to generalize" were an operant response class. Based on the research that has been reviewed, there are many common components that are suggested as helpful when trying to increase generalization of skills being taught to children. It is important to include diverse types of reinforcement, natural ! #(! reinforcement if possible, peers to teach, self-monitoring, teaching in the "real world" setting, and using multiple teaching examples. These strategies may be useful individually, and even more so if combined into the training program. Promising Approaches to Make Social Skills More Effective Video-Modeling Interventions Video-modeling was incorporated into the Superheroes Social Skills program as one component to help increase its effectiveness. Video self-modeling is implemented by having the target child watch a video of themselves performing the desired skill without error, whereas video-modeling is the process of watching a video of a peer demonstrating appropriate use of the skill or behavior. The child is shown the videos repeatedly and this has resulted in changes in behavior, maintenance, and generalization. Hitchcock, Dowrick, and Prater (2003) reviewed 18 research studies that included 129 participants and used video self-modeling as an intervention. The behaviors that were targeted by the video-modeling interventions included academics, compliant classroom behavior, peer relationships, disruptive behavior, adaptive behavior, and language. Overall, results indicated that video self-modeling was an effective intervention for changing behavior, improving academics, and increasing communication for children. Bellini et al. (2007) found that children with ASD demonstrated increased social engagement that was maintained over time as the result of video self-modeling. Bellini, Akullian, and Hopf (2007) also found that video self-modeling not only increased the effectiveness of social skills training, but that the effects were maintained after the ! #)! intervention was completed. The maintenance of positive results following treatment, as well as generalization to other individuals and settings was also found by Sherer et al. (2001). Nikopoulos (2007) found increased interaction time and generalization of play skills to new toys and settings for children with ASD after viewing video-modeling of typically developing peers. The generalization of play was also maintained for up to three months. Charlop-Christy and Danshevar (2003) concluded that generalization increased when using video-modeling and believe that the video stimulus is reinforcing and possibly helpful in controlling overstimulation for children with ASD because the video presentation helps to focus the attention on one stimulus. Research has found video-modeling is more effective than in-vivo modeling and it is also a cost effective alternative to other forms of training (Bellini & Akullian, 2007; Miller, 2006). While there are many studies that have indicated high effects of video self-modeling, there have also been studies that compare video self-modeling to video-modeling. Results indicate that both forms of modeling produce moderate to large effect sizes, suggesting there is little difference in effectiveness between them (Bellini et al., 2007; Sherer et al., 2001). Thus, video-modeling, either self or peer, is now considered an effective and important component of social skills training. Peer Mediated Interventions The use of peer mediation in social skills interventions for children has been used to counteract the poor generalization of social skills taught through didactic instruction delivered by adults (Rogers, 2000). Studies have concluded that peer mediated programs ! $+! are an effective way to teach social skills; however, researchers have found the effects are difficult to maintain because children tend to rely on the peer cues and prompts (McConnell, 2002; Rogers, 2000). Miller's (2006) meta-analysis indicated that peer mediated interventions are the most effective for school age children with ASD when learning social behaviors. Schneider, Goldstein, and Parker (2008) completed a meta-analysis on single-subject research studies conducted with children with autism. Using the percentage of non-overlapping data points (PAND) as a measure of effect, peer mediation was found to be an effective treatment to teach social skills to children with ASD. Chen (2006) conducted a review of multiple research studies conducted with children with emotional and behavior disorders. The research focused on various strategies for teaching social skills to children and found that peer mediation was an effective component to teaching social skills to children with emotional and behavior disorders. It was naturally reinforcing to children to have peers reinforce their behavior, as opposed to adults. It was also more natural for them to generalize the skills without relying on the prompts they would have received from adults. The research supporting the use of peers as tutors or in helping teach children social skills is increasing. Many of the social skills programs currently available do not use peer mediation as part of their instruction, but this may prove to be a component that is useful in the development of future social skills training programs. ! $*! Self-Management Interventions Self-management is used to teach children to monitor and record their own behavior by increasing their awareness of the behavior and their use of the behavior in multiple and unsupervised settings. Stahmer and Schreibman (1992) implemented self-management interventions to children with ASD in order to increase appropriate play behaviors. They found that self-management increased the use of appropriate play, decreased self-stimulatory behaviors, and that the results were maintained and generalized to unsupervised settings. Koegel, Koegel, Hurley, and Frea (1992) found similar results when teaching self-management to children with ASD in an attempt to increase social responsiveness and decrease disruptive behavior. Chen (2006) conducted a review of studies conducted with children with emotional and behavior disorders. The research focused on different strategies for teaching social skills and found that self-monitoring was an effective component to teaching social skills to children with emotional and behavior disorders. Lewis, Hudson, Richter, and Johnson (2004) identified self-management as a scientifically supported practice for use with children with emotional and behavioral disorders. Generally, social skills training was not found to be effective based on the meta-analysis completed, but self-management was still an effective component of social skills. Based on Strain, Kohler, and Gresham (1998), social skills and self-management may be even more effective than they were found to be in this meta-analysis. Their conclusions are such that single-subject research studies are not appropriate for meta-analyses, which may indicate different effects if analyzed differently. ! $"! The self-management training had the desired effects on the individuals and it generalized to multiple settings (school, home, and community) without the treatment provider present. Self-management is another component that has been found to be effective for children and may prove to be an essential component of social skills training programs. Social Stories Social stories have also been studied as an effective component of social skills training. Social stories are stories created to reflect realistic situations that might require the use of skills being taught and demonstrating how these skills can be used appropriately in various social situations. Social stories also often include pictures, which can be helpful for children with ASD who benefit from the use of visual formats. Quirmbach, Lincoln, Feinberg-Gizzo, Ingersoll, and Andrews (2009) found that the use of social stories significantly improved play behavior for children. Hagiwara and Myles (1999), however, did not find consistent and significant results for the participants in their study; rather, the effects were only found for outlier participants. However, for the participants that Hagiwara and Myles found benefited from the intervention, the effects generalized to other situations and could be linked to the skills. Social stories have been shown in the research to be an effective strategy for teaching social skills, but may not be as effective when used as the only form of intervention (Crozier & Tincani, 2007; Sansoti, Powell-Smith, & Kincaid, 2004). It is likely helpful to combine this intervention technique with others when developing social skills programs. ! $#! Ravary, Unesi, and Looye (2008) conducted a study with 22 females between the ages of 10 and 11 who were displaying problem behaviors. This study concluded that a story-based social skills intervention successfully reduced symptoms of conduct disorder for the participants. Hanley-Hochdorfer, Bray, Kehle, and Elinoff (2010) studied the effects of using social stories to increase the pro-social verbalizations of children with ASD. They found that social stories, when used alone as an intervention, were not able to produce large effects. The researchers concluded that social stories should be used with caution as the sole intervention procedure, but that they are far more effective when used in conjunction with other interventions. The meta-analysis by Kokina and Kern (2010) also found questionable effects of using social stories as the only intervention method. They concluded that social stories were more effective at minimizing disruptive behaviors than increasing pro-social behaviors and teaching social skills. One hypothesis for this outcome that was presented by the researchers was that children might lack the knowledge and training of the social skills necessary to follow the social skills in the social story. This may suggest that by teaching the social skills in tandem with using the social stories, the effects might be greater. Increasing Generalization with "Stickiness" In the book The Tipping Point, Malcolm Gladwell discusses the concept of epidemics and how certain concepts are made to stick with people. In the chapter entitled The Stickiness Factor, he discusses advertising techniques and television shows that have ! $$! found a way to be sticky to their audience. He posits, "if you paid careful attention to the structure and format of your material, you could dramatically enhance stickiness" (p. 110). By increasing the stickiness of what you are teaching, you would also increase the application of the concepts introduced past the immediate environment of exposure. Stickiness can be defined as an attribute of a stimulus that enhances its maintenance across situations and time. Gladwell uses different advertising techniques as examples of stickiness, but he also examines children's television shows that attempted to increase children's literacy, such as Sesame Street and Blues Clues. The research that was done on the effects of these television shows indicates that there are certain factors that are essential to achieve stickiness with children. Repetition is one of the key components of stickiness because if a concept is repeated to children, they are able to remember the information and recall it better at a later time. The content should also be creative, so as to draw the child's attention. Another factor that is essential to stickiness for children is to make the presentation of the material appealing (e.g., using muppets and animation). It was also helpful for these television shows to include an interactive element that allowed children the opportunity to answer questions or guess missing information. In an attempt to appeal to children and make the information sticky for them, it would be important to include some or all of these elements. The stickier the presentation of the material is, the more likely children are to recall this information at later times. This can be applied to any information being taught to children, including ! $%! social skills training. As Gladwell states, "There is a simple way to package information that, under the right circumstances, can make it irresistible" (p.132). Heath and Heath (2008) also lay out a framework for making information "sticky." In their book, Made to Stick, they describe a method of SUCCESs: a simple unexpected concrete credentialed emotional story. These factors all help to increase how "sticky" information is when presented. In their book, they use the idea of how "sticky" urban legends are and what makes them this way. They then identify the aspects of urban legends that can be used to increase the "stickiness" of other information that is presented to people. Simplicity means that the information must be profound, as well as simple. Unexpectedness in the information presented helps to generate interest and curiosity, while making the information concrete in the messages and images makes it easier to remember. The information must seem credible and allow for people to test the ideas for themselves. It is also helpful to make people feel something about the information they are receiving. Stories can also help increase their memory of the information. By using these strategies, it is implied that information can be made "sticky" for the people exposed to the content. If it is "sticky," they will be more likely to recall the information at a later time. The Incredible Years Program (Webster-Stratton, 1984) is a popular social skills curriculum that has incorporated some of these strategies and has been proven to be effective in the research (Taylor, Schmidt, Pepler, & Hodgins, 1998; Webster-Stratton & Hammond, 1997; Webster-Stratton, Reid, & Hammond, 2004). The Incredible Years programs use videos that are watched multiple times to make the material repetitive. The group members discuss the content of the video ! $&! vignettes in order to provide an interactive component. For the children's lessons, life-sized puppets are used, which makes the material appealing. This is an example of a program that has incorporated some of the elements of "stickiness" in order to effectively teach social skills to children. Superheroes Social Skills The Superheroes Social Skills Program (Jenson, et al., 2011) was developed based on past research in order to incorporate many previously discussed evidence-based components of existing social skills programs into one program. Some of the components that are used in this program include video modeling with an optional video self-modeling component, peer mediation through the inclusion of typically developing "peer buddies", self-management of the child's use of the learned skills, and social stories in the form of comic books. The program was developed to teach social skills to children with Autism, Asperger's Disorder, or Pervasive Developmental Disorder - Not Otherwise Specified, but due to the evidence-based practices combined within this program, it may be effectively used with other populations, including children with high incidence disabilities. Many existing programs are effective, but lack maintenance effects and generalization of the skills that were taught. One of the main goals of this program is not only to effectively teach social skills to children, but for the skills to be generalized and maintained. Block (2010) found the effects of Superheroes Social Skills to generalize to recess and the effects were not only maintained at a 2-week follow-up but they were actually increased. ! $'! Superheroes Social Skills includes 18 skills separated into foundational, intermediate, or advanced skills based on the complexity of the skill. Each of the 18 skills is typically taught twice per week, but only 11 of these skills that were considered the most important skills for the children with high incidence disabilities were selected for this study. The skills selected for use in this study were identified to be the most applicable and pertinent to children with high incidence disabilities. The skills taught were: 1. Introduction/Get Ready 2. Following Directions 3. Anxiety Reduction 4. Participate 5. Body Basics 6. Expressing Wants and Needs 7. Turn Taking 8. Recognizing Emotions 9. Perspective Taking 10. Bullying 11. Problem-Solving Skills are introduced by the superheroes (The Initiator and Interactor Girl) and their sidekick (Scooter the robot) in an animated video at the beginning of the lessons. The lesson format also includes role-playing social scenarios by pairing children with high incidence disabilities and their peer buddies. Participants then watch a digital comic social story with a hard copy provided to the participants at the end of the lesson. The ! $(! video animation and comic books make this program of high interest for children, but while still incorporating evidence-based components to encourage skill acquisition, maintenance, and generalization by the children. In addition to these components, the lessons include social games that reinforce the skills being taught in an enjoyable format. Reinforcement strategies to encourage rule-following behavior and compliance are used throughout the lessons (see Appendix A). The goal in the development of the Superheroes Social Skills program was to incorporate multiple components that have met the criteria for evidence-based practice. This study has incorporated several evidence-based practices including modeling and video-modeling, peer mediation, self-management procedures, social stories, and direct instruction. The use of multiple evidence-based practices in one program makes the potential for efficacy favorable when compared to other social skills programs. These same components are also likely to overcome some of the shortcomings of other existing programs, including generalization and maintenance of social skills. The effectiveness of the Superheroes Social Skills program (Jenson, et al., 2011) for use with children with Autism Spectrum Disorders has already been studied by members of the development team. Block (2010) completed research in a public school with four children with high-functioning autism and found large effect sizes during free play observations (M=0.85) and during recess observations (M=2.34). It was also measured that the effects of the social skills training were still present at a two-week follow-up for the free play observations (M=0.74) and the recess observations (M=3.42), indicating that the effects were not only maintained, but continued to increase following treatment. This study also indicated that the program did not increase social initiations ! $)! and sometimes the participants actually decreased their use of initiations in interactions, but social responses were consistently and greatly increased. Radley (2010) completed a research study with preschool children with Autism Spectrum Disorder at a specialized school for children with Autism. He provided the video-based version of the program to one group and taught the program material without using the videos for the other group. His results indicated large effect sizes for both the didactic group (M=1.54) and the video-based group (M=0.93). One of the students in the video-based group was fairly nonresponsive to the treatment and one student in the didactic group was extremely responsive to the treatment, which may account for the higher mean effect size for the didactic group. Hood (2010) completed research at an out-patient clinical setting with four elementary-aged children with Autism Spectrum Disorder. The lessons were combined and taught once per week in the evenings. The results of the free play observations indicated an overall large effect size for the group (M=1.07). Parents also completed a daily report of their child's use of the skills at home as a measure of generalization. The mean effect size as measured by the daily report was 1.13. Overall, these studies have indicated that the Superheroes Social Skills program is an effective program for teaching social skills to children with Autism Spectrum Disorder. The current study will expand this research to determine if it is also an effective program to teach social skills to children with high incidence disabilities in a school setting. ! %+! Summary In summary, there are many programs that have been developed to aid children in learning and using social skills, but many have been found to have little, if any, effect. Despite the small effects of social skills programs currently being taught, many schools and clinical settings still provide social skills training due to the negative outcomes that are associated with children with poor social skills. It is necessary for programs to be developed and research to be conducted in order to identify evidence-based social skills programs for children. The Superheroes Social Skills program (Jenson, et al., 2011) has been developed to meet these criteria by combining many of the components of other programs that have been proven effective in the research. Along with the evidence-based practices, the Superheroes Social Skills program also uses video animation and other high-interest media to increase the children's level of interest, attention to the material, and generalization of the skills. The present study was conducted to evaluate if the Superheroes Social Skills program is an evidence-based approach to teaching social skills when delivered as a pull-out group intervention in a school setting for four children with high-incidence disabilities. Efficacy was determined by calculating effect size and percentage of nonoverlapping data points (PND) for each participant, as well as an effect size and percentage of all nonoverlapping data points (PAND) for the group. ! %*! Statement of Purpose This study was designed to evaluate the use of the Superheroes Social Skills program (Jenson, et al., 2011) as an evidence-based practice to teach social skills to children with high-incidence disabilities in a pullout group in a school setting. The purpose of this program is to provide children with the appropriate social skills necessary for participation in pro-social interactions with peers and adults. Another goal of this study is to measure generalization of the skills to multiple situations, such as recess, and maintenance of these skills at a 2-week follow-up. The program is based on a superhero theme with animation and comic books as high interest media to appeal to participants and maintain attention to program content throughout the intervention. The program also incorporates research-validated components, such as video-modeling, social stories, peer mediation, and self-management strategies. The program effectiveness was determined by increased use of social skills during free play observation periods following the lessons, observation of increased pro-social behaviors in a generalized recess setting, and completion of checklists including the BIRS, SSIS, and the Children's Consumer Satisfaction Survey. ! %"! Research Questions The following research questions were addressed in this study: 1. What is the effectiveness of the social skills intervention during a free play observation? Following each session, there were free time play periods that were video-taped and then coded for social behaviors using an adapted partial interval observation system (see Appendix B). The data were then used to calculate effect sizes, PND, and PAND. 2. What is the effectiveness of the social skills intervention as measured by the spontaneous generalized use of pro-social behaviors during recess? Each participant was observed four times throughout treatment during recess. The observations were video-taped and coded for social behaviors using an adapted observation system. The data were then used to calculate effect sizes, PND, and PAND. 3. What is the maintenance of pro-social behaviors at a 2-week follow-up? There were two free play and two recess observations completed for each participant conducted two weeks after treatment was completed. The observations were video-taped and coded for social behaviors using an adapted observation system. The data were then used to calculate effect sizes, PND, and PAND. 4. What is the consumer satisfaction with the intervention? Parents and teachers completed the Behavior Intervention Rating Scale (BIRS) (see Appendix C) following the last treatment session and descriptive statistics were used to analyze the parent satisfaction of the treatment. 5. What is the effectiveness of the intervention based on the results of the Social Skills Improvement System (SSIS)? ! %#! Parents and teachers completed the SSIS as a pre- and posttest measure and standard deviation changes in scores were used to determine treatment effects. 6. What is the social validity of the intervention? Parents and teachers completed an adapted version of the Social Validity Scale (see Appendix D) (Bellini, 2006). Descriptive Statistics were used to analyze the social validity of the intervention. 7. What is the participant satisfaction with the intervention? Participants completed a child consumer satisfaction survey (see Appendix E) and descriptive statistics were used to determine the consumer satisfaction of the intervention. 8. What amount of progress was made regarding the performance and demonstration of target social skills by student participants as measured by the Superheroes Social Skills Progress Monitoring Tool over the span of 11 role-play scenarios? Participants were observed role-playing the lesson's skill and steps during each lesson. A percentage for the skill steps that were appropriately demonstrated was calculated. CHAPTER 2 METHODS This study was designed to evaluate the use of the Superheroes Social Skills program (Jenson et al., 2011) as an evidence-based practice to teach social skills to children with high-incidence disabilities in a pullout group in a school setting. The purpose of this program is to provide children with the appropriate social skills necessary for participation in pro-social interactions with peers and adults. Another goal of this study is to measure generalization of the skills to multiple situations, such as recess, and maintenance of these skills at a 2-week follow-up. The program is based on a superhero theme with animation and comic books as high interest media to appeal to participants and maintain attention to program content throughout the intervention. The program also incorporates research-validated components, such as video-modeling, social stories, peer mediation, and self-management strategies. The program effectiveness was determined by increased use of social skills during free play observation periods following the lessons, observation of increased pro-social behaviors in a generalized recess setting, and completion of checklists including the BIRS, SSIS, and the Children's Consumer Satisfaction Survey. Prior to recruitment of participants, consent to conduct the research study by the school district institutional review board and the school was obtained. The primary ! ""! researcher also obtained approval from the University of Utah Institutional Review Board. Participants This study was conducted with 4 children with high-incidence disabilities and 4 peer buddies between the ages of 5 and 9 who were nominated by school staff. The researcher contacted the parents of the children who were nominated and they were given more detailed information by phone. If the parent wanted to have their child participate in the program, they met with the researcher to complete the parental consent (see Appendix F) and child assent forms (see Appendix G) and the parents of the children with high-incidence disabilities also completed the SSIS and placement checklist during this initial meeting. All of the children with high-incidence disabilities were required to meet the following inclusion criteria. In order to be included as a participant, children had to meet the following criteria: 1. Have a current medical diagnosis of conduct disorder, anxiety disorder, learning disability, or ADHD by a physician, psychologist, or psychiatrist or an educational classification of emotional disturbance, specific learning disability, speech/language impairment, or other health impairment. 2. Obtain scores on the SSIS and another behavioral measure that meet criteria for having a significant behavioral or social impairment. 3. Obtain a verbal IQ score of 70 or higher on a standardized intelligence test, administered within the past 3 years by a qualified administrator. ! "#! 4. Possess and demonstrate use of sufficient expressive and receptive language so as to be able to participate in the social skills group. In addition to meeting these criteria, a placement checklist (see Appendix H) designed for this study to screen participants was administered to parents to aid in the selection of participants. Parents of the participants attended an initial parent training session to be informed about the homework, skills to be taught, and the lesson format. Table 5 provides a summary of the participant characteristics and is followed by a more detailed description of the individual participant characteristics. Participant 1 is a 5-year-old Caucasian male with a diagnosis of Attention Deficit/Hyperactivity Disorder from a physician. His cognitive ability was assessed in preschool using the Stanford-Binet Intelligence Scales, Fifth Edition. He earned a Full Scale IQ standard score of 98, a Nonverbal IQ of 97, and a Verbal IQ of 98. Based on parent ratings on the SSIS, participant 1 was rated below average (SS=67) on the Social Skills Scale and above average (SS=115) on the Problem Behaviors Scale. His teacher rated him below average (SS=86) on the Social Skills Scale and average (SS=109) on the Problem Behaviors Scale. On the Behavior Assessment System for Children, Second Edition, his mother rated him in the at-risk range (SS=66) on the Externalizing Problems Scale. These scores indicate there are significant deficits in the areas of social skills and problem behaviors and he may benefit from intervention in these areas. Participant 1 is above grade level academically, but is significantly below grade level socially and behaviorally. His teacher reported that he is frequently yelling out in class, has difficulty keeping his hands to himself, does not respect personal space, and he has a difficult time ! "$! Table 5 Participant Characteristics Demographic Information for Participants Participant 1 Participant 2 Participant 3 Participant 4 Mean Child's Age 5.6 7.1 9.1 8.11 7.48 IQ Scores FSIQ 98 95 89 80 90.5 NVIQ 97 101 91 79 92 VIQ 98 90 86 83 89.25 SSIS (Pre-intervention) (Social Skills Scale) Parent 67 83 77 98 81.25 Teacher 86 81 76 93 84 (Problem Behaviors) Parent 115 93 117 116 110.25 Teacher 109 117 121 126 118.25 BASC-II (Externalizing Problems Score) Parent 66 58 79 54 64.25 Teacher 88 53 70.5 Conners - 3 (Aggression Score) Parent --- 68 122 55 61.5 Teacher 68 104 66 79.3 Conners - 3 (Hyperactivity/Impulsivity Score) Parent --- >90 69 68 79 Teacher 89 97 65 83.6 ! "%! initiating appropriately with his peers. His parents reported that he is very smart, but has a lot of energy and needs help focusing. Participant 2 is a 7-year-old male with an educational classification of speech/language impairment. He had not been administered a cognitive assessment prior to this research study. The primary researcher administered the Stanford-Binet Intelligence Scales, Fifth Edition. He earned a Full Scale IQ standard score of 95, a Nonverbal IQ of 101, and a Verbal IQ of 90. Based on parent ratings on the SSIS, participant 2 was rated below average (SS=83) on the Social Skills Scale and average (SS=93) on the Problem Behaviors Scale. His teacher rated him below average (SS=81) on the Social Skills Scale and above average (SS=117) on the Problem Behaviors Scale. On the Behavior Assessment System for Children, Second Edition, his mother rated him in the average range (SS=58) on the Externalizing Problems Scale. On the Conners Rating Scales, Third Edition, his mother rated him in the elevated range (SS=68) in the area of Aggression and in the very elevated range (SS>90) in the area of Hyperactivity/Impulsivity. These scores indicate there are significant deficits in the areas of social skills and problem behaviors and he may benefit from intervention in these areas. Participant 2 is below grade level academically, socially, and behaviorally. His teacher reported that he does not complete assignments, teases other children, and has difficulty making friends. His mother reported that he has difficulty following directions at home. Participant 3 is a 9-year-old male with a medical diagnosis of Attention Deficit/Hyperactivity Disorder by a physician. He was administered the Stanford-Binet Intelligence Scales, Fifth Edition. He earned a Full Scale IQ standard score of 89, a ! "&! Nonverbal IQ of 91, and a Verbal IQ of 86. Based on parent ratings on the SSIS, participant 3 was rated below average (SS=77) on the Social Skills Scale and above average (SS=117) on the Problem Behaviors Scale. His teacher rated him below average (SS=76) on the Social Skills Scale and average (SS=121) on the Problem Behaviors Scale. On the Behavior Assessment System for Children, Second Edition, his mother rated him in the clinically significant range (SS=79) on the Externalizing Problems Scale. These scores indicate there are significant deficits in the areas of social skills and problem behaviors and he may benefit from intervention in these areas. Participant 3 is below grade level in academics and behavior, but on grade level socially. His teacher reports he does not complete assignments and he sometimes lies and steals. His mother reports that he is defiant at home. Participant 4 is an 8-year-old male with an educational classification of speech/language impairment. He was administered the Stanford-Binet Intelligence Scales, Fifth Edition. He earned a Full Scale IQ standard score of 80, a Nonverbal IQ of 79, and a Verbal IQ of 83. Based on parent ratings on the SSIS, participant 4 was rated average (SS=98) on the Social Skills Scale and above average (SS=116) on the Problem Behaviors Scale. His teacher rated him average (SS=93) on the Social Skills Scale and above average (SS=126) on the Problem Behaviors Scale. On the Behavior Assessment System for Children, Second Edition, his mother rated him in the average range (SS=54) on the Externalizing Problems Scale. On the Conners Rating Scales, Third Edition, his mother rated him in the average range (SS=55) in the area of Aggression and in the elevated range (SS=68) in the area of Hyperactivity/Impulsivity. These scores indicate there are significant deficits in the areas of social skills and problem behaviors and he ! #'! may benefit from intervention in these areas. Participant 4 is below grade level academically, socially, and behaviorally. His teacher reports that he gets frustrated and aggressive toward other students, he has tried to choke two students, and he does not complete tasks in class. His mother reports that he gets distracted easily, but he does not show frustration or aggression at home. Four peer buddies were nominated by school staff to participate in the intervention groups. !()*!+,-./01!-**2!-02+./.-03+4!2035*6!.3!05*4!7289!05*!4.:!+8!05*! 3.3*;!!<7!+)*!78=2!+,-./01!-**2!-02+./.-03+4>!83*!?04!7*901*!036!+)2**!?*2*!901*;!!@8! +,-./01!-**2!-02+./.-03+4!?*2*!/=22*3+1,!/1044.7.*6!?.+)!0!6.40A.1.+,!.3!4-*/.01! *6=/0+.83;!!All of the peer buddies were nominated by staff and identified as having appropriate social skills. Staff also nominated all of the peer buddies as having high average academic performance in class to ensure their participation in the groups did not affect their school performance.! Setting The sessions of the social skills program were conducted at an elementary school in an urban school district with approximately 40,000 students. All sessions took place in the school psychologist's office that contained one desk, a smartboard and ceiling projector for the videos to be played on, and a rug with color squares for the children to sit on. The analog free time play was conducted in the same room as the intervention sessions. Toys available to the children during free play included LEGOS (LEGO), Ants in the Pants Spongebob Squarepants Edition (Hasbro), Don't Break the Ice (Hasbro), toy ! #B! cars with a track (Mattel), Transformers (Hasbro), and Jenga (Parker Brothers). The 10- minute observation periods during free play were videotaped for coding and reliability purposes. The treatment sessions were also videotaped to ensure treatment integrity. The recess observations were conducted on one of the two playgrounds located at the school. One playground is surrounded by a cement sidewalk and a fence. The ground of the playground area is covered by woodchips and contains two plastic tubes to crawl in, three large plastic animals to climb on, and a large playground system. The second playground includes a field with soccer goals, a wood-chipped playground area with a large playground system, and a large cement area with basketball hoops, a kickball diamond, and painted foursquare areas. Dependent Measures Observation System An observation system (see Appendix B) was adapted from Bellini's Social Observation System (2007) and Ross and Horner (2009). The observation system was used to code behaviors during the videotaped 10-minute free play periods during baseline, following each treatment session, and at a 2-week follow-up. It was also used to code the videotapes from the baseline, treatment, and follow-up recess observations for each participant. Children were coded for using the following behaviors: positive initiations, positive responses, physical aggression, verbal aggression, and neutral behavior. Bellini's Social Observation System provides codes for the areas of positive initiations and positive responses. Codes for verbal aggression and physical aggression were ! #C! adapted from Ross and Horner's (2009) definitions, and neutral behavior was added as a behavior code. Positive initiations were defined as appropriately and positively initiates some form of interaction, such as: request assistance, request information, request interaction/participation, provide a greeting or compliment, giving, sharing, showing, offer comfort/physical affection, positively and independently joins play activity/interaction. Positive responses were defined as appropriately and positively responds to an initiation by someone else, including: provides assistance, responds to request/provides information, joins activity when asked, responds to greeting/compliment, responds to physical affection, responds well when others start a conversation/activity, stays calm when teased, responds positively to criticism, positively participates in games or group activities, responds positively or appropriately when pushed or hit. Verbal aggression was defined as directing verbal or gestural negative communication toward one or more children, including: teasing, taunting, threatening, negative body language, and negative gestures. Physical aggression was defined as an act of negative and/or inappropriate physical contact with another person (behaviors within games were considered physical aggression when they went beyond the expectations of the game), such as: hitting, biting, kicking, choking, stealing, throwing objects, restricting freedom of movement, and physically forces others to act against their will. Neutral behavior was defined as taking part in an activity without having any interaction with others (e.g., solitary play, parallel play). The observation system uses a 10-second partial interval recording method of observing behaviors, in which the observer watches the behavior for 5 seconds, and then records the first behavior observed as a positive initiation, positive response, physical ! #D! aggression, verbal aggression, or neutral behavior during the next 5 seconds. The observations for free play periods and recess were all videotaped and then the observations were reviewed and coded by the researcher and 33% of the observations were coded by another graduate student separately. The coding was then compared using Kappa after both observers had completed their coding to determine interrater reliability. The formula that was used to calculate Kappa is K = (Po - Pe) / (1-Pe). Po is the observed proportion of agreement and Pe is the proportion of agreement expected by chance. Kappa was calculated by entering each raters codes into a website (http://cosmion.net/jeroen/software/kappa/). Social Skills Improvement System The Social Skills Improvement System (SSIS) (Gresham & Elliott, 2008) is a rating scale that measures the domains of social skills, problem behaviors, and academic competence. There are separate teacher rating forms and parent rating forms that were completed pre- and posttest. On each item, the rater indicated how frequently the child performed the behavior (Never, Seldom, Often, Almost Always). The ratings were then transferred to a corresponding number (0 = Never, 1 = Seldom, 2 = Often, 3 = Almost Always) and these scores were then converted into standard scores and percentile ranks. The assessment was used to help identify target behaviors and measure progress throughout the treatment based on standard deviation changes between scores, as suggested in the manual. ! #E! Behavior Intervention Rating Scale (BIRS) The BIRS is considered to be a valid measure of treatment acceptability and effectiveness (see Appendix C). The BIRS was administered to the parents of participants following the completion of the intervention. Parents rated questions about the effectiveness of the treatment on a six-point scale. Ratings range from 1 to 6, which indicate the parent and teacher strongly disagrees, disagrees, somewhat disagrees, somewhat agrees, agrees, or strongly agrees. The means were calculated for each item and used to determine the level of treatment acceptability. Social Validity Checklist The social validity of the intervention was evaluated using a social validity scale that has already been developed and tested for its psychometric properties. The Social Validity Checklist was developed by Bellini (unpublished) and was adapted for use in this study (see Appendix D). Parents completed the checklist after the last intervention session by responding to five questions about the program's effectiveness. Possible answers on the scale range from strongly disagree to strongly agree. Answers were then given a numerical value (Strongly Disagree = 1, Disagree = 2, Agree = 3, Strongly Agree = 4). The total possible score for each item is 4 and the total possible score for the scale is 20. The means were calculated for each item and used to determine the treatment's level of social validity. ! #"! Child Consumer Satisfaction Survey A child consumer satisfaction survey was administered to the participants and the peer buddies following the intervention in order to determine the acceptability of the treatment from the children's perspective. The Child Consumer Satisfaction Survey (CCSS) was developed for use in this study (see Appendix E). Questions were read aloud to the children and then they circled the answer they felt was most accurate. There are four possible choices for responses to the questions (Strongly Disagree, Disagree, Agree, Strongly Agree). Means were calculated based on the responses and were used to determine the participants' perceptions of the Superheroes Social Skills program. Treatment Fidelity Checklist A checklist was created in order to assess the level of fidelity in implementation of the program (see Appendix I). Each step of the lesson implementation was listed on a form and following each session, the researcher indicated which steps were implemented by marking the checklist. A second graduate student reviewed 33% of the videotaped sessions and marked the number of steps followed correctly to ensure reliability. A percentage of successful step implementation was then calculated by dividing the number of steps implemented by the total number of steps for each lesson. All of the treatment fidelity forms were totaled after the completion of all sessions to obtain a mean treatment fidelity percentage. ! ##! Design Data analysis was completed using a replicated AB single-subject design (Harris & Jenson, 1985). Participants were observed during analog free play periods for three baseline sessions, after the eleven intervention sessions, and twice at a 2-week follow-up. The children were also observed at recess for three baseline observations, four observations during the treatment phase, and twice at a 2-week follow-up. Single-subject research has been used to study the effectiveness of various interventions. Kazdin (1992) stated that single-subject research could be used to draw valid inferences about interventions as long as continuous observations are completed prior to treatment, during the baseline phase, and throughout the treatment phase. Baseline observations are used to determine a trend in the baseline and establish stability. This trend can be compared to treatment observations to determine if the intervention had an effect on the projected trend. Internal threats of validity exist in this type of single-subject design, including maturation, testing effects, and history threats. Historical confounding could also be a possible threat, but is minimized with more than one subject and frequent observations. According to Kratochwill (1978), threats of maturation are minimized if repeated measurement is used, threats of history can also be minimized, and threats of testing effects are minimized if there is not repetitive exposure to a pretest. Specifically, AB designs with replication are found to control for historical threats to internal validity if subjects are exposed to multiple and variable environments during the treatment period (Harris & Jenson, 1985). ! #$! There are many threats to internal and extern