Twenty-first century community learning centers' teacher perspectives on the academic achievement of students with disabilities

The purpose of this study was to examine the relationship between days and hours of supplementary reading and math instruction provided to students with disabilities that participated in a 21st Century Community Learning Center (21st CCLC) and their regular classroom teachers' perception of their progress toward reading and math proficiency. The study analyzed 294 school records of students that attended a 21st CCLC in Colorado and New Mexico during the 2007-2008 program year. Records analyzed for this study included students (a) receiving special education services during the regular education day, (b) who spent the majority (more than 50%) of their school day in a regular education classroom setting, and (c) receiving special education or related services as identified in an individualized education program (IEP). The results of the study suggest that a statistically significant relationship exists between the hours of reading and math instruction that students with disabilities receive in a 21st CCLC and their regular classroom teacher's perception of progress towards proficiency. The study results support a need for changes to the current evaluation methods of programs participating in 21st Century Community Learning Centers. Data utilized for this study were taken from a convenience sample and conclusions regarding its generalizability (external validity) are limited. Recommendations for future evaluation methods are suggested and future research in the area is discussed.

TWENTY-FIRST CENTURY COMMUNITY LEARNING CENTERS' TEACHER PERSPECTIVES ON THE ACADEMIC ACHIEVEMENT OF STUDENTS WITH DISABILITIES by Joe Max Lang A dissertation submitted to the faculty of The University of Utah in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Special Education The University of Utah December 2012 Copyright © Joe Max Lang 2012 All Rights Reserved The University of Utah Graduate School STATEMENT OF DISSERTATION APPROVAL The dissertation of Joe Max Lang has been approved by the following supervisory committee members: Michael Hardman , Co-Chair 6-13-12 Date Approved Patricia Santistevan Matthews , Co-Chair 6-13-12 Date Approved Clifford Drew , Member 6-13-12 Date Approved Susan Johnston , Member 6-13-12 Date Approved Thomas Hudson , Member 6-13-12 Date Approved and by John McDonnell , Chair of the Department of Special Education and by Charles A. Wight, Dean of The Graduate School. ABSTRACT The purpose of this study was to examine the relationship between days and hours of supplementary reading and math instruction provided to students with disabilities that participated in a 21st Century Community Learning Center (21st CCLC) and their regular classroom teachers' perception of their progress toward reading and math proficiency. The study analyzed 294 school records of students that attended a 21st CCLC in Colorado and New Mexico during the 2007-2008 program year. Records analyzed for this study included students (a) receiving special education services during the regular education day, (b) who spent the majority (more than 50%) of their school day in a regular education classroom setting, and (c) receiving special education or related services as identified in an individualized education program (IEP). The results of the study suggest that a statistically significant relationship exists between the hours of reading and math instruction that students with disabilities receive in a 21st CCLC and their regular classroom teacher's perception of progress towards proficiency. The study results support a need for changes to the current evaluation methods of programs participating in 21st Century Community Learning Centers. Data utilized for this study were taken from a convenience sample and conclusions regarding its generalizability (external validity) are limited. Recommendations for future evaluation methods are suggested and future research in the area is discussed. TABLE OF CONTENTS ABSTRACT ....................................................................................................................... iii LIST OF TABLES ............................................................................................................. vi Chapters I. INTRODUCTION AND REVIEW OF LITERATURE .........................................1 History of Out-of-School-Time Programs ...............................................................2 Historical Studies of OST Programs ........................................................................5 Changes to the 21st CCLC under NCLB ..................................................................9 Contemporary Studies of the 21st CCLC ...............................................................18 Statement of the Problem .......................................................................................23 Research Questions ................................................................................................24 II. RESEARCH DESIGN AND METHOD ...............................................................26 Participant Selection Criteria .................................................................................27 Study Design ..........................................................................................................27 Study Procedures ...................................................................................................28 Statistical Methods .................................................................................................31 Administrative Responsibilities .............................................................................31 III. RESULTS ..............................................................................................................33 Student Participation by 21st CCLC Site and Grade ..............................................33 Data Analysis .........................................................................................................37 IV. DISCUSSION ........................................................................................................43 Review of the Problem...........................................................................................43 Similarities and Differences Among Data Collection Sites ...................................44 Discussion of Study Results...................................................................................47 Study Limitations ...................................................................................................53 Implications for Practice ........................................................................................54 Implications for Future Research ...........................................................................58 v Summary ................................................................................................................59 REFERENCES ..................................................................................................................62 LIST OF TABLES Table Page 1. Demographics by 21st CCLC Site ..............................................................................35 2. Demographics by Grade Level ...................................................................................38 3. Spearman Rank-Order Correlations Between Number of Days of Attendance and Reading Grades, Teacher Ratings, and PPICS Teacher Surveys in Reading ......40 4. Spearman Rank-Order Correlations Between Number of Days of Attendance and Math Grades, Teacher Ratings and PPICS Teacher Surveys in Math .................40 5. Spearman Rank-Order Correlations Between Number of Hours of Reading Instruction and Reading Grades, Teacher Ratings, and PPICS Teacher Surveys in Reading ...................................................................................................................41 6. Spearman Rank-Order Correlations Between Number of Hours of Math Instruction and Math Grades, Teacher Ratings, and PPICS Teacher Surveys in Math .......................................................................................................................42 CHAPTER I INTRODUCTION AND REVIEW OF LITERATURE In the United States, the current education system has increasingly become focused on student academic achievement. New federal regulations mandate that states increase test scores for all students, including historically low performing subgroups of students. Schools utilize additional programs to supplement and expand regular education services to help make the mandated state test score benchmarks. One program that has been supported politically and financially by the Federal Government is the 21st Century Community Learning Centers (21st CCLC). Twenty-First Century Community Learning Centers are out-of-school-time programs designed to expand student academic activities and provide for community needs in which they are located (No Child Left Behind Act (NCLB) of 2001: Purpose; Definitions, 2008). They began as federally and privately funded out-of-school-time (OST) programs designed to keep youth off the streets during hours when their parents were working. As public and political interest and ensuing funding for these programs has increased, so has the academic focus of the activities provided in them and the subsequent requirements of the entities funding them. The No Child Left Behind Act (NCLB)(2001) has created an atmosphere in which the 21st CCLC are being called upon to align their services and coordinate with the regular school day activities in schools, 2 districts, and states. This is being accomplished to ensure that statutorily mandated academic benchmarks and goals are met and associated sanctions are avoided. As the 21st CCLC program intent grows closer to meeting NCLB goals, research regarding 21st CCLCs' effect on the academic achievement of their participating students is expanding. However, research on students with disabilities as a specific subgroup participating in 21st CCLC is nonexistent. This lack of research on students with disabilities' academic progress on state core standards is critical to the need for substantiated, research-based continuation of 21st CCLC as a viable option for supplementary educational services. History of Out-of-School-Time Programs Out-of-school-time (OST) programs are designed for school-aged children when they are not in school. These programs have been around for many years, first receiving federal funding after World War II when women entering the workforce were looking for childcare (Coleman, 1995; Miller, Snow, & Lauer, 2004; Rose, 1999). Later, public perceptions of an increased geographic concentration of poverty in the inner cities of the United States resulted in OST programs gaining momentum from the 1960s through the 1980s (Miller et al., 2004). During this time, community interest groups expressed a need for OST programs, especially during after-school hours. This was due in large part to perceived increases in sexual activity among youth, teen pregnancy, drug and alcohol abuse, and gang activity. It was noted that most of these activities were occurring between the hours of 3:00 and 6:00 p.m. when parents were still at work and children were left to their own devices (Moore, Driscoll, & Lindberg, 1998). 3 Political and public interest in OST programs continued to grow throughout the 1990s as a response to the increased need for high-quality, affordable day care because of a notable increase in the female labor force (Seppanen, DeVries, & Seligson, 1993). Educational analysts have pointed to four principle factors driving public perceptions of the need for OST programs, especially for after-school programs. These include (a) a belief that typical public spaces are no longer safe places for children to be in during after-school time, (b) a perception that it is stressful and unproductive for children to be left alone after-school, (c) concerns that many children need increased, individual educational attention in order to master academic skills, and (d) a belief that low-income children should have the same opportunities as their more advantaged peers (Halpern, 1999). The historic constant for OST programs is that they be designed for children from low-income families. OST programs have their roots in a diverse mix of public interests and private and government investments. They have usually been unevenly distributed across and within communities where they are being implemented (Ianni, 1989). Many of these OST programs were funded by government sources while others were funded by charitable contributions focused to eliminate identified or perceived problems in specific communities. The federally created, 21st Century Community Learning Centers has become the most monetarily supported OST program to date. In 1994, the Elementary and Secondary Education Act of 1965 was reauthorized as the Improving America's Schools Act of 1994. It sanctioned the 21st Century Community Learning Centers Act under Title X, Part I. Authorized for 5 years and administered by the United States Department of Education, the 21st Century Community 4 Learning Centers (21st CCLC) provided federal grants directly to schools, consortia of schools, or other community entities for OST programs. A legal requirement of receiving funds was that they be disseminated equitably among states for rural and inner-city communities. Programs were awarded grants for 3 years and had to include at least 4 of 13 projected activities intended to serve their local communities. The 13 projected activities included such things as education, day care, health, social services, recreation, and other community improvement activities (Improving America's Schools Act, 1994). Political support for the 21st CCLC under the Clinton Administration is evident by its congressional funding appropriation. In their first year of funding, 21st CCLC were appropriated $750,000, $1 million in 1997, $40 million in 1998, and $200 million in 1999. By 2000, 21st CCLC were appropriated $845.6 million dollars (United States Department of Education, 2007). As 21st CCLC grew, they shifted focus from programs whose intent was to offer a broad selection of services for their communities to programs that included an absolute priority for providing activities that expanded learning opportunities (McCallion, 2003). Public and political interest continued for OST programs in general. However, the dramatic increase in funding and mandated focus on academic services for the 21st CCLC also created calls for research efforts to determine whether the 21st CCLC were actually effective in providing the services for which they were designed. As an example, in a hearing before the subcommittee on Early Childhood Youth and Families of the One Hundred and Sixth Second Session of Congress, Chairman Michael Castle expressed concerns regarding the increase in funding and subsequent desire to have research regarding the program's effectiveness saying the following: 5 This [21st CCLC] is growing faster than Intel's stock is growing. Let me just say, parenthetically, I am going to have some questions about this. I could not be a stronger supporter of this program, but I never thought it was going to happen quite like this. I have serious questions about that (Examining the 21st Century Community Learning Centers Program, 2000). The conception of OST programs was initially driven by public, political, and private perceptions of broad community needs. Initial research and studies regarding them was mostly descriptive in nature. Historical Studies of OST Programs Seppanen, DeVries, and Seligson conducted a study entitled the National Study of Before- and After-School Programs. The study served to "establish a descriptive foundation upon which others can build additional research to inform policy and practice" (Seppanen et al., 1993). Though this study provided valuable information on a national scope regarding types of programs that existed and where they were being held, it did not answer the questions regarding whether the programs were actually achieving the academic goals they had established for the families and children in the communities in which they were operating. There was no disaggregation of student data mentioned in relation to the academic impact the OST programs had on subgroups of students (Seppanen et al., 1993). Subsequent studies began to dig deeper in OST programs' effects on students, but still fell short of providing substantial, disaggregated data on participating subgroups of students. In 1995, the UCLA Center for the Study of Evaluation conducted a widely cited longitudinal study of LA's Better Educated Students for Tomorrow (LA's BEST) after-school programs. LA's BEST programs were designed for schools where student test scores were lower than the state averages, where overall school population socio-6 economic status was consistently lower than the surrounding community, and where crime rates had shown increases over the preceding years. Participants in the study were recruited from the 10 longest running LA's BEST sites, including students in fifth, sixth, and seventh grades who had participated in the program for 2 or more years. Using students' grade point averages and various student and parent questionnaires as measures for determining program impact on student success, results demonstrated that participants' grades improved and general attitudes towards school improved during the course of the 2-year study (Brooks, Mojica, & Land, 1995). Though the LA's BEST study made attempts toward determining whether program efforts were effective in achieving its goals, mainly student academic achievement, it did not disaggregate data results according to demographic student subgroups or end-of-level testing results. In 1998, Fashola authored a report called "Review of Extended-Day and After-School Programs and Their Effectiveness." The report explicitly evaluated 34 existing extended day and after-school programs, "in terms of their evidence of effectiveness for improving student outcomes and their evidence of replicability in other locations" (Fashola, 1998). All programs reviewed in Fashola's report were designed for students at risk for academic failure. The review broke the 34 programs into five categories depending upon their overarching academic focuses. These five categories included (a) language arts programs, (b) study skills programs, (c) academic programs in other skills areas, (d) tutoring programs for reading, and (e) community-based programs. The Fashola report is of particular interest because it recognizes the lack of prior research on the academic achievement results for OST programs. It also identified common designs and practices among successful programs and offered specific 7 recommendations regarding the implementation of an extended day and after-school program that would ultimately be successful for achieving academic goals for at risk participating students. Specifically, the Fashola report provided strong recommendations for successful after-school academic programs, summarizing that those programs must "have clear goals, well developed procedures for attaining those goals, and extensive professional development." These recommendations are significant in light of the direction in which 21st CCLC after-school program requirements would follow under the No Child Left Behind Act of 2001. It is important to note that the Fashola report does not disaggregate study subjects by subgroup. In the midst of the significant increases in federal funding for OST programs, a second national study was conducted in 1999 specifically for 21st CCLC. The study investigated a variety of negative and positive research report findings on 21st CCLC sites from around the country. In an effort to do this, the U.S. Department of Education contracted with Mathematica Policy Research, Inc., and Decision Information Resources, Inc., to evaluate the 21st CCLC on a national scope. The study gathered data for 2 reporting years, the 2000-2001 and the 2001-2002 school years. It should also be noted that the grantees from which data were used for this study were in their second or third year of a 3-year grant period. None of the study participants for the Mathematica Policy Research, Inc., evaluation were selected from state-administered programs (Dynarski, James-Burdumy, Moore, Rosenberg, Deke, & Mansfield, 2004). Subsequent studies on the 21st CCLC would be conducted on state-administered programs only. For the first year of data collection, 2000-2001, the investigators gathered data on elementary programs based on random assignment from 1,000 participating students in 8 18 schools from seven school districts. Students participating in the study were randomly assigned to either the treatment (21st CCLC participants) or the control group (non - 21st CCLC participants) for year-to-year comparisons. In the second year of the evaluation, 2001-2002, the researchers expanded their elementary school samples to 2,308 students in 26 schools in 12 districts (Dynarski et al., 2004). The evaluation of the middle school program participants was done as a matched- comparison design, contrasting student outcomes from those who participated in the 21st CCLC program to those students who did not in the same reporting period. The middle school study looked at 4,300 middle school students in 61 schools in 32 school districts. This portion of the evaluation differed from the elementary piece in that it did not take into consideration new grantees; rather, it added more data gathered on the students from the first reporting year through surveys from teachers, students, and parents to be used for comparison between the first and second years of the study (Dynarski et al., 2004). Data for the final report for elementary 21st CCLC programs were collected through student, teacher, principal, and program staff questionnaires, as well as Stanford Achievement Tests - Version 9 (SAT-9), school records, program attendance records, and site visits. The outcomes of interest from the Mathematica Policy Research, Inc., and Decision Information Resources evaluation are the academic achievement results that could be measured through an established academic test score. Results from SAT-9 reading scores for elementary school programs indicated that the 21st CCLC made no impact on treatment students' scores: treatment students demonstrated SAT-9 reading scores of 35.0 (in percentiles) while control students scored 35.9 (in percentiles). The Mathematica Policy Research, Inc., and Decision Information 9 Resources evaluation concluded that similar no-impact results were demonstrated for the six student and parent characteristic-based subgroups for whom SAT-9 reading score data was collected. Race and ethnicity were disaggregated for the report; however, students with disabilities were not identified as a subgroup for the elementary program portion of this evaluation (Dynarski et al., 2004). For the middle school 21st CCLC programs, the evaluation did not report conclusions based on any standardized test results. Rather, it reported data based on student, teacher, principal, and program staff questionnaires which largely focused on perceptions of student progress towards program goals. Student grades were compared between the treatment group and the control group. However, results were inconclusive and led evaluators to conclude that the 21st CCLC had little impact on student academic achievement. As was the case for the elementary school 21st CCLC, the Mathematica Policy Research, Inc., and Decision Information Resources evaluation disaggregation of data for students identified for services under special education was not included. Changes to the 21st CCLC under NCLB During the Mathematica Policy Research, Inc., and Decision Information Resources study, the Elementary and Secondary Education Act (1965) was reauthorized in 2001 as the No Child Left Behind Act (NCLB) (2001). The 21st CCLC were reauthorized under Title IV, Part B and amended as 21st Century Schools. The purpose of the 21st CCLC, as defined under NCLB (2001), "is to provide opportunities for communities to establish or expand [student] activities in community learning centers" (No Child Left Behind Act of 2001: Purpose; Definitions, 2008). A crucial requirement regarding the purpose of the reauthorized 21st CCLC is that their focus be that of 10 academic enrichment (No Child Left Behind Act of 2001: Purpose; Definitions, 2008). One critical change under the NCLB (2001) is that funding for the 21st CCLC be allotted to state educational agencies (SEA) instead of directly to individual program sites. This change, therefore, places SEA in charge of awarding subgrants to 21st CCLC sites and to monitor their effectiveness (No Child Left Behind Act of 2001: Allotments to States, 2008). Under the NCLB (2001), state grantees are required to give an assurance in their application to the Department of Education that the proposed program will serve students that primarily attend schools that are eligible for schoolwide Title I, Part A funds (i.e., schools with the highest poverty levels); this is pursuant to Section 1114 of the No Child Left Behind Act of 2001 (No Child Left Behind Act of 2001: Local Competitive Grant Program, 2008). Additionally, SEAs are required to give granting priority to local 21st CCLC sites that will serve students attending Title I schools that have been identified as in need of improvement under the No Child Left Behind Act (2001), Section 1116 (No Child Left Behind Act of 2001: Local Competitive Grant Program, 2008). The trend of subgranting to 21st CCLC sites based in public schools has increased under the No Child Left Behind Act (2001). For example, by 2005, 68% of 21st CCLC program centers, fiscal agents, or grantees were school districts. During the 2003-2004 reporting period 90% of the 21st CCLC program centers were located in public school buildings (Mitchell, Naftzger, Margolin, & Kaufman, 2005). By placing state administration of the 21st CCLC in the hands of the SEA, the NCLB (2001) imposed the cooperation and coordination of schools and their 21st CCLC sites to achieve their respective goals for their student populations; schools are now looking to 21st CCLC to help them achieve the challenging academic goals. 11 The collaboration of 21st CCLC sites and their regular school day counterparts has become a necessary component of a child's educational experience. In light of this, an explanation of the accountability mandates to which schools, districts, and states must adhere is warranted. The NCLB (2001) mandates extensive accountability for schools, districts, and states that receive federal Title I money regarding student academic performance as determined by scores on state and district-wide assessments (No Child Left Behind Act of 2001: State Plans, 2008). The law requires that schools make adequate yearly progress (AYP) toward a minimum benchmark of proficiency on the state assessment, culminating in a proficiency level of 100% by the 2013-2014 school year (No Child Left Behind Act of 2001: State Plans, 2008). In order for a school to make AYP, the overall school student population and each student subgroup must make benchmarked academic progress toward achieving state standards in both reading and mathematics. The racial groups reported in the measurement of academic progress are Caucasian, African American, Hispanic, Native American, Asian, and Pacific Islander. The subgroups are multiracial, economically disadvantaged students, limited English proficient, and students with disabilities (No Child Left Behind Act of 2001: Academic Assessment, 2008). This is true in a similar but expanded manner for districts and the state (Olson, 2004). As part of making AYP, schools must test at least 95% of the whole student school population and 95% of the students in each subgroup. If any one student group in reading or math at a school does not perform at the target goal percentage, the school is considered to have not made AYP for that year, unless safe harbor conditions are met. AYP by means of safe harbor is defined as a subgroup making progress on an academic 12 indicator reducing the percentage of students not proficient by 10% from the previous year (No Child Left Behind Act of 2001: State Plans, 2008). A school is identified for sanctions after it has not made AYP for 2 consecutive school years (No Child Left Behind Act of 2001: State Plans, 2008). A school moves on to increasing levels of sanctions if it continues not to make AYP. The timeline and succession of sanctions for schools that fail to make AYP is as follows: schools that do not make AYP for 2 successive years receive a "school in need of improvement" designation and must receive technical assistance to improve. The district must offer the opportunity for students to transfer to another public school (No Child Left Behind Act of 2001: Academic Assessment, 2008). Schools that do not make AYP for 3 consecutive years will continue activities from the first year of school improvement and the district must use additional Title I funds for Supplemental Education Services (SES). The state must issue a list of approved providers, which may include the district or outside groups, including for-profit companies or community-based organizations. This particular requirement becomes critically important as districts attempt to unify funding and look to 21st CCLC programs as a resource to help them make AYP. The culmination of sanctions for schools that do not make AYP includes required and overarching corrective action plans for the school, extending the school day or year, or replacing school staff. After 6 years of failure to make AYP, a school is faced with the required sanction of converting the school into a charter school, replacing all of the staff, or turning the school over to a private management company (No Child Left Behind Act of 2001: Academic Assessment, 2008). NCLB (2001) implements extensive measures of accountability for schools, districts, and states regarding the participation of all students 13 (i.e., student population overall and defined subgroup student populations) in state academic assessments. For students with disabilities as a subgroup, NCLB permits states to develop alternate academic achievement standards, but only for students with the most significant cognitive disabilities. In general terms, almost every student that is identified for special education services is required to take the state's standard academic achievement test and have their scores included in the AYP determinations for schools, districts, and states (No Child Left Behind Act of 2001: Academic Assessment, 2008). Ensuring that students with disabilities make AYP is challenging. Schools, districts, and states face a conflict in having them included in AYP determinations (McLaughlin, Embler, & Nagle, 2004). States are working to assist local districts and schools to make the continuously escalating benchmarks for AYP. However, where budgets are not definite, administrators will need to find ways in which to combine funds and align programs in order to achieve better outcomes toward student academic achievement as defined by the No Child Left Behind Act (2001). The Federal Government has made 21st CCLC programs a viable option for schools wishing to use it as an academic interventional resource through large funding appropriations. All projections show that 21st CCLC programs will continue to be funded for years to come, making them a continued attractive option for schools to supplement educational opportunities for low performing students, i.e., students with disabilities (Office of Management and Budget, 2007). The Department of Education is also encouraging educational programs to coordinate services. An example of the Department of Education's focus on encouraging program coordination is that states are being given freedom to approve 21st CCLC sites as either 14 Supplemental Education Service (SES) providers themselves, or to be used in conjunction with other state approved SES providers (Office of Innovation and Improvement, 2008). Supplemental Educational Services are provided for students in schools that have been identified as being in need of improvement under the NCLB (2001). They refer to free academic assistance to students in reading, language arts, and math. Supplemental Education Services can be provided through tutoring, remediation, and other educational interventions offered outside the regular school day hours. State educational agencies are responsible for identifying and approving SES providers. The corresponding districts make available a list of those providers to the parents of students that would be eligible to receive those services under Title I, Part A (No Child Left Behind Act of 2001: Academic Assessment and Local Educational Agency, 2008). This option for schools, districts, and states furthers the alignment of 21st CCLC to the No Child Left Behind Act (2001) goals and gives precedence to those programs as a resource for schools needing to target specific academic areas for low performing groups of students (e.g., students with disabilities). Any educational program that is federally funded through the U.S. Department of Education, including 21st CCLC, is required to articulate what the program is attempting to accomplish in regard to the performance indicators provided through the Government Performance and Results Act of 1993. Government Performance and Results Act (1993) performance indicators are measured so that program progress can be reported to Congress (Office of Management and Budget, 2007). In that respect, states receiving 21st CCLC funding must utilize the Profile and Performance Information Collection System (PPICS). This mandate has been in effect since 2004. 15 The PPICS is a web-based system funded by the U.S. Department of Education and designed by Learning Point Associates (U.S. Department of Education Office, 2004). PPICS is designed to meet four principle purposes: (a) to obtain the data to report on the indicators for the 21st CCLC program in accordance with the Government Performance Results Act of 1993 (Government Performance Results Act, 1993); (b) to obtain information that will allow the U.S. Department of Education to monitor how the program is operating under state administration; (c) to provide the Department of Education staff with the capacity to respond to congressional Office of Management and Budget and other departmental inquiries about the program; and (d) to provide state 21st CCLC staff with a series of system-supported reports and related features that facilitate their ability to use data to assess the performance of grantees in their state, and to inform related monitoring, evaluation, and technical assistance efforts (Fortune, 2006). Data are collected in PPICS through four modules. The first module, called the Competition Overview, reports basic descriptive information from states about the outcomes of a given subgrant competition (e.g., number of applicants, number of grants awarded). Also, information is given about the performance indicators and priorities employed at the state in structuring its statewide program. Data are entered directly online via a chart of Fiscal Agent Types (e.g., school district, community-based organization, nationally affiliated nonprofit agency, etc.). On the Fiscal Agent Types chart, the number of applications the state received from each Fiscal Agent Type and the number of actual local program awards that were granted is reported (Learning Point Associates, 2008). 16 The second part of the Competition Overview asks the grantee to check all performance indicators that the state selected in its application to the Department of Education. The grantee also needs to check which indicators the individual grantees checked for compliance to the SEA. There are eight options that grantees can check: (a) student achievement on standardized tests, (b) student classroom performance, (c) student attendance during the regular school day, (d) student attendance in the 21st CCLC Program, (e) student behavior, (f) graduation rates, (g) student satisfaction with Center activities and services, and (h) parent satisfaction with center activities and services (Learning Point Associates, 2008). There is also a place for the state to describe any other performance indicators for which it has decided to collect data. The state is then asked to identify how they classified the high-poverty schools in the request for proposal (RFP). Finally, the grantee is required to identify the items that were formally recognized in the RFP to the Department of Education as a priority. They either indicate them as mandatory, optional, or not a priority. There are 14 priorities from which to identify the three levels and an option to specify other. The 14 priorities in PPICS include such things as identifying the type of center, the focus group of students, e.g., elementary, middle school, or high school, and specifications of the types of services that will be provided at the center. These funding priorities have a significant influence on the programming objectives, structural program features, and ultimately, program evaluation results (Naftzger, Margolin, Kaufman, & Ali, 2006). The second module, referred to as the Grantee Profile Module, is meant to provide information about what is presently true about a given grantee's 21st CCLC operations, or if a grantee has not yet begun operations, what the grantee intends to do in 17 the way of service provision. Also reported are the objectives and activities local grantees propose to deliver at each of their centers as well as the students and family members they intend to serve (Fotune, 2006). Each active grantee in this section is asked to provide information in four sections: Basic Information, Objectives, Partners, and Centers. These sections collect such data as award amounts, center location, contact personnel, description of the project, and supplementary sources of funding. Of note regarding academic data, Section 2 asks grant recipients to choose from a list of provided objectives and select those that apply to their specific 21st CCLC site (Learning Point Associates, 2008). Module 3 is identified as, "State Activities." State Education Agencies serve as the fiscal agents for 21st CCLC and it is the state's responsibility to suballocate these funds to the different grantees within the state. Module 3 outlines the process and procedure in which State Education Agencies allocate their current-year allocations to local 21st CCLC programs (Fortune, 2006). Lastly, the Annual Performance Report (APR) is the data collection module most important to this proposed study. The APR module purposes are (a) to collect data from 21st CCLC grantees on progress made during the preceding year in meeting their project objectives, (b) to collect data on what elements characterized center operation during the reporting period, including the student and adult populations served, and (c) to collect data that address the Government Performance and Results Act (1993) performance indicators for the 21st CCLC program (Fortune, 2006). Only those grantees that are in at least the second year of funding and have submitted APR data complete this section of the modules. In these sections, grantees report the current status of the project objectives 18 identified in the Grantee Profile (Learning Point Associates, 2008). On the PPICS website, status of objectives can be identified as (a) met the objective, (b) did not meet, but progressed, (c) did not meet and did not progress, (d) unable to measure progress, (e) revised the objective, (f) dropped the stated objective, or (g) objective not associated with this reporting period. In light of the reported status of program objectives, four general categories of Government Performance Results Act (1993) performance indicators have been specified by the Department of Education. These categories must be utilized to assess the extent to which the 21st CCLC are meeting their statutorily authorized purposes on a national basis. The Government Performance Results Act (1993) indicators specify requirements and target performance levels in terms of (a) improvement in student achievement, (b) improvement in student academic behavior, (c) the condition of academic enrichment offerings, with a special emphasis on enrichment opportunities in technology, and (d) the degree to which centers emphasize one or more core academic areas. Each performance indicator is a measure of the effectiveness of the 21st CCLC site in achieving one of their goals (Office of Management and Budget, 2007). The implementation of PPICS as a data collection mechanism for the 21st CCLC has provided a central location for the United States Department of Education to conduct evaluations. Contemporary Studies of the 21st CCLC In 2005, Learning Point Associates in a grant funded through the U.S. Department of Education produced the first national evaluation for the 21st CCLC since the federal funding changed to state administration. The report was based on the data extracted from the PPICS web-based data system and is titled 21st Century Community Learning Centers 19 (21st CCLC) Analytic Support for Evaluation and Program Monitoring: An Overview of the 21st CCLC Program. The purpose of the evaluation was to provide a descriptive look at the 21st CCLC state-administered program from a national perspective (Mitchell, Naftzger, Margolin, & Kaufman, 2005). Of particular interest are the results outlined in Section 6: Student Achievement and Behavioral Outcomes. Section 6 specifically describes the extent to which the reporting sites progressed toward the Government Performance Results Act (1993) targeted performance levels and the performance indicators identified for the 21st CCLC. The extent to which grantees reported that they were accomplishing the objectives they identified relating to improving student achievement and academic behaviors was also detailed in this section. Through PPICS, the Learning Point Associates report used seven measures to evaluate progress towards the Government Performance Results Act (1993) indicators. Those measures are (a) the percentage of regular student 21st CCLC program participants (i.e., those who attended the program 30 days or more) whose grades in math or English improved from fall to spring, (b) the percentage of regular attendees whose state achievement test scores moved from "not proficient" to "proficient" or above during the 2003-2004 Annual Performance Report reporting period, (c) the percentage of regular program participants whose teachers reported through PPICS that they had improved in classroom participation and homework completion, (d) the percentage of regular 21st CCLC sites' participants whose teachers reported their classroom behavior had improved, (e) the percentage of 21st CCLC sites that reported at least one of their emphases was in a core academic content area, (f) the percentage of 21st CCLC sites that reported offering enrichment activities in technology, and (g) the percentage of 21st CCLC sites that 20 reported offering activities or enrichment and support in other areas (Mitchell et al., 2005). The Learning Point Associates reported on the regular attendees' academic performance based on federal proficiency levels (i.e., Basic, Proficient, and Advanced) in reading, language arts, and mathematics for the 2003-2004 reporting period (Mitchell et al., 2005). The cross-year changes in proficiency levels from 2002-2003 were also provided for the 2003-2004 reporting period in reading and language arts and mathematics. Overall, about 45% of regular attendees scored below proficient in reading and language arts and 49% scored below proficient in mathematics. For cross-year changes, 31% of regular attendees showed increases in both reading and language arts and mathematics while 20% of regular attendees witnessed a decrease in both academic areas (Mitchell et al., 2005). It is important to note that 21st CCLC grantees self-identified site objectives from a selection menu and self-indicated the category that they indicated best described them. Grantees also provided information as to whether their 21st CCLC site objectives had been met or whether progress had been made toward their goals (Mitchell et al., 2005). No disaggregated data were reported on any participating students with disabilities' performance on academic indicators. In July of 2006, Learning Point Associates published the second report regarding the performance of the 21st CCLC on a national basis from PPICS data collected for the 2004-2005 reporting period. The report is titled 21st Century Community Learning Centers (21st CCLC) Analytic Support for Evaluation and Program Monitoring: An Overview of the 21st CCLC Program: 2004-05 (Naftzger, Margolin, Kaufman, & Ali, 21 2006). This report is again a descriptive look at the nature and scope of the 21st CCLC from a national perspective. The 2006 Learning Point Associates report detailed the achievement of 21st CCLC sites relative to the performance indicators that were self-reported. It also provided information regarding how sites changed over time. Analysis of the existing 9,930 sites across the nation demonstrated that more than 90% focused their federally mandated core academic activities on reading and mathematics. They correlated their areas of emphasis with those of NCLB. Ninety percent of the centers provided tutoring and homework help, 84% of centers offered academic enrichment learning programs, and 74% offered academic improvement and remediation programs. Another important difference between the 2005 report and the 2006 report was an effort to look at student demographics by disaggregated subgroups. The subgroups included Native American, Asian, Black, Hispanic, White, Limited English Proficient (LEP), students receiving free or reduced lunch, and students with disabilities. However, the 2006 Learning Point Report details only numbers of regular 21st CCLC attendees that were categorized into the above mentioned subgroups (Naftzger et al., 2006). It does not report their performance data collected through the Annual Performance Report section of the PPICS system. Though cross-year comparisons from the 2005 and 2006 reports that the percentages of students that moved from basic to proficient on state tests in language arts and mathematics increased, it only reported these data from five participating states and did not give a demonstration of results from the disaggregated student subgroups. More precisely, if students participated in the 21st CCLC and were receiving services through special education services under IDEA 2004, the PPICS 22 requested only that total, aggregated numbers of such students be reported. There was no mention of the impact of the 21st CCLC on the academic achievement of students who are also served in special education programs and that had an Individual Education Program. States receiving 21st CCLC grants must participate in data collection through the PPICS as a means to conduct a comprehensive evaluation of how programs as a whole are progressing towards the performance indicators and goals as described in its application to the Department of Education. The State Education Agency's local 21st CCLC sites must also be periodically monitored and evaluated by the SEA. The results of those evaluations must be disseminated to the public (No Child Left Behind Act: State Application, 2008). Those evaluations are under the administration and development of the individual SEA. They may be done by the states, provided for through a grant to another agency, or contracted out to a third party evaluator from state set-aside 21st CCLC funds (Non-Regulatory Guidance for 21st Century Community Learning Centers, 2003). The legal requirement of the SEA local 21st CCLC site evaluations is based on the Principles of Effectiveness as outlined in NCLB. The Principles of Effectiveness require that program activities be based on an assessment of objective data regarding the need for the program and activities in the communities and schools. It must also be based upon the established set of performance measures as defined in the state's application for 21st CCLC funds from the Department of Education. The SEA evaluations must also have their foundation focused upon scientifically based research practices that will help participating students achieve state academic achievement standards (No Child Left 23 Behind Act: Local Activities, 2008). The evaluation tools used to conduct local program evaluations are chosen by, or approved by, the SEA. Statement of the Problem The 21st Century Community Learning Centers began as child-care centers for working parents. As public and political interest grew in 21st CCLC, so did federal funding and a call for clearly defined 21st CCLC direction and subsequent goals and research. Under No Child Left Behind (2001), the 21st CCLCs' purpose and requirements have been more closely aligned with those of Title I, Part A. Now under state administration, the 21st CCLC have begun to be called upon by schools, districts, and states to help them ensure that all students achieve academic progress towards AYP benchmarks and goals. The 21st Century Community Learning Centers are currently required to participate in the PPICS national data collection system and be monitored by their administrating SEA. However, options given to states on what specific categories of student data are collected and reported create an inconsistent forum for making research-based judgments regarding how well 21st CCLC are impacting student academic achievement under NCLB requirements. This is due to the fact that each state chooses its program emphasis differently. The same problem occurs for evaluations of local grantees because their academic progress is being evaluated against the categories and achievement benchmarks that the SEA chose in the application for funds through the PPICS. 24 The performance of the "students with disabilities" subgroup under NCLB has not been included in studies of 21st CCLC effectiveness on student academic achievement and is not reported through the PPICS. Specifically, there is no research on the impact of 21st CCLC site participation for students receiving special education services under IDEA 2004. There is no research on students' progress towards state standards and NCLB or AYP benchmarks for student academic achievement. Current studies involving students with disabilities' performance in 21st CCLC sites are mostly descriptive in nature and based on participant perceptions as reported by data collected from 21st CCLC stakeholder questionnaires and surveys regarding their satisfaction with the programs. Research Questions The focus of this study is to investigate relations between days and hours of supplementary reading and math instructional services in a 21st CCLC site and the academic progress of participating students with disabilities in the subject areas of reading and math who are spending the majority of their day in a regular classroom setting. The null-hypothesis is that there is no correlation between the days or hours of study in a 21st CCLC site and the academic progress of students with disabilities. This study is exploratory in design. The study utilized a convenience sample of data obtained from the third-party evaluator Educational Research and Training Corporation (ERTC). The purpose of this study was to provide insight into the current relationship between the academic achievement of students with disabilities and their participation in 21st CCLC. The two research questions to be addressed in the present study are the following: 25 1. What is the correlation in Colorado and New Mexico between the number of days that students with disabilities spend in 21st CCLC and their regular classroom teachers' perceptions of student progress toward reading and math proficiency on state standards? 2. What is the correlation in two states, Colorado and New Mexico, between the number of hours that students with disabilities spend in 21st CCLC receiving reading and math instruction and their regular classroom teachers' perceptions of student progress toward reading and math proficiency on state standards? CHAPTER II RESEARCH DESIGN AND METHOD This exploratory study responded to two research questions in order to assess if the amount of time that students with disabilities who attended 21st Century Community Learning Centers (21st CCLC) in two states had an effect on their academic achievement. Students in the study were enrolled in New Mexico and Colorado during the 2007-2008 school year. The study was aimed at determining if enrollment in the 21st CCLC had an impact on academic achievement in their regular classroom. These two states were selected for this study because the 21st CCLC reported substantial numbers of participating students with disabilities who had relevant data reported on their academic status for program evaluation to a third- party contractor, Educational Research and Training Corporation (ERTC). This study assessed if the study participants' regular classroom teachers perceived notable changes in those students' progress toward proficiency on state standards in reading and math during the program span. Study participants attended one of the 21st CCLC from Colorado or New Mexico during the 2007-2008 school year and were reported by the 21st CCLC as belonging to the subgroup, "student with disabilities." The participating students received reading and math instruction in one of the 21st CCLC.27 Study data were accessed through an existent electronic data-file that was reported by individual 21st CCLC administration and staff and collected by the third-party evaluator, ERTC. ERTC collected these data both to assist these 21st CCLC in completing federal and state reporting requirements and to provide an evaluative report individualized to each program site. ERTC provided the data collection instruments and teacher ratings to each 21st CCLC. The data from the identified subgroup, students with disabilities, were extracted from the database as the focus of the study and then aggregated as a subgroup for analysis. Participant Selection Criteria Data were collected on students in Colorado and New Mexico who had attended a 21st CCLC and had been identified as receiving special education services during the regular education day and receiving services under an Individualized Education Program (IEP). As such, the data for this study were selected using convenience sampling (Drew, Hardman, & Hosp, 2008). For identified students, data were accessed for 17 21st CCLC sites located in either Colorado or New Mexico. There were 294 student records examined from 21st CCLC in Colorado and New Mexico during the 2007- 2008 school year. Of the 294 records, 162 students (55%) attended an elementary school program (grades k - 5), 113 students (38%) attended a middle school program (grades 6 - 8), and 19 students (6%) attended a high school program (grades 9 - 12). Study Design This study is exploratory by design in order to investigate relationships in the data collected from the convenience sample (Drew, Hardman, & Hosp, 2008). The study 28 utilized a correlational method. This method was selected to determine if a relationship (correlation) existed between time spent in a 21st CCLC and the academic achievement of special education students (Drew, Hardman, & Hosp, 2008). Correlation between academic achievement and time receiving reading and math instruction in 21st CCLC programs was determined through three formats of data: (a) a state-standards-based teacher rating of student performance, (b) 21st CCLC program federally mandated teacher surveys, and (c) student grades. Additional academic data were collected by each 21st CCLC for participants (e.g., state assessment scores); however, these data were not available for use in this study. The three aforementioned formats of data were utilized for this study because permission for their use was granted by the New Mexico Public Education Department and the Colorado Department of Education. Also, the above three formats of data addressed this study's research questions regarding the academic achievement of students with disabilities and their participation in 21st CCLC. Study Procedures To provide for data regarding study participants' academic achievement, regular day teachers completed a reporting instrument in three different formats both at the beginning and end of the 2007-2008 school year. Through onsite coordinators at each 21st CCLC, regular day teachers were instructed to record the specified student information in an electronic data-file, which was stored by Educational Research and Training Corporation (ERTC). All onsite coordinators were trained in using a four-point rubric for teacher ratings, student grades, and PPICS teacher surveys. Data collected included demographics, test scores, teacher ratings, grades, hours attending the after- school program and hours of specific instructional services provided (e.g., reading and 29 math tutoring and homework help). Data for this study were collected from this convenience sample maintained by ERTC. ERTC facilitated the coding of this convenience sample and provided the data file for this study. Only data for students identified as receiving special education services during the regular school day and working under an IEP were extracted for this study. All student identifiers were eliminated from these files. Teacher Rating Rubric One instrument this study used to determine teachers' perception of student progress toward proficiency on state standards included student data collected through a teacher rating rubric. Regular education teachers were asked to rate performance, based on their opinion of student academic performance focusing on state core standards. The rubric consisted of a four-point rating scale (1= satisfactory, 2 = partially proficient, 3= proficient, and 4 = advanced). Teachers were trained by ERTC on the expected use and scoring protocol of the rubric. All personnel who participated in the 21st Century teacher ratings of student proficiency in relation to state content standards completed a minimum of 3 hours of training designed to enhance rating reliability and validity. Participating teachers were provided with materials describing how and when to rate students and were provided with detailed descriptions of what each rating on the four-point rubric meant. Teachers were asked to rate students twice each program year (once in late September at the beginning of the academic year and once in late May at the end of the year). Teachers were asked to leave ratings blank if they were unsure of a particular student's proficiency. Survey participants, at a minimum, were asked to rate students in reading and math. Teachers were instructed to only rate students for whom they had primary 30 instructional responsibility and to not guess at proficiency: they were to base their rating on a variety of evidence. This evidence included other assessment scores, student work, and teacher judgment regarding proficiency based on participation in class. For the teacher rating forms, teachers were asked to disregard students' grades and base their ratings solely on student performance in relation to state standards. Participants were given practice in how to rate students during this training through role playing exercises and were guided in rating example students. This was followed up by examination of each teacher's ratings and discussion of discrepancies to ensure consistent ratings across the group. Classroom Grades In addition to the teacher rating rubrics, students' classroom grades were collected for reading and math. Grades were collected twice during the school year, once in the fall of 2007 and again in the spring of 2008. All grades collected were converted into a rubric score with zero being the lowest score (F) given and four being the highest (A). Teacher Surveys Lastly, the PPICS teacher surveys were collected for every student in reading and math. The PPICS surveys contained additional teacher responses, including such things as attendance, attitude, and behavior. However, for this study, only the survey responses regarding teachers' perceptions of student academic growth in reading and math were used. These surveys were collected one time in the spring of 2008. The PPICS Likert scale responses were converted into a code to indicate to what extent students' regular 31 classroom teachers perceived changes had occurred regarding students' academic performance in reading and math during the school year. Statistical Methods A statistical Spearman's rank-order test was conducted to ascertain correlations between the numbers of days that students attended a 21st CCLC program and their regular classroom teachers' perception of progress towards academic proficiency in reading and math. In other words, was there a relationship between the numbers of days that a student with a disability attends a 21st CCLC and perceived increases in student academic proficiency demonstrated in the regular classroom (Pett, 1997)? A Spearman's rank-order test was completed for the group of teacher responses as a whole and then again independently for teacher ratings, classroom grades, and the PPICS teacher surveys. Each was done for both reading and math. In similar fashion, a Spearman rank-order correlation test was run to determine if any correlations existed with respect to the numbers of hours that students with disabilities received reading instruction and math instruction in one of the 21st CCLC. Administrative Responsibilities I obtained permission from ERTC to access all required data records. Permission was also acquired from the Colorado Department of Education as well as the New Mexico Public Education Department. I, in cooperation with each state's education department, ensured that no student names, numbers, or identifiers, aside from specifics from the data collected, were disseminated or reported. Upon completion of the study, all 32 data files were saved in a password protected computer at the investigator's private residence.CHAPTER III RESULTS Chapter 3 summarizes the data collected by ERTC on students with disabilities from Colorado and New Mexico who participated in a 21st Century Community Learning Center. All data collected for this study included students identified as receiving special education services in their schools during the regular school day. Data were collected during the 2007-2008 year from 17 sites in Colorado and New Mexico. Data were reported by 21st CCLC site administrators in the same format for elementary (grades K - 5), middle school (grades 6 - 8), and high school programs (grades 9 - 12). The 3 forms of data reported were state-standards-based teacher ratings of student performance, 21st CCLC federally mandated teacher surveys, and student grades scaled to a uniform reporting rubric. Demographic data are organized by student participation by site and student participation by grade. Statistical data were organized by research question (a) the number of days students attended and academic achievement, and (b) the numbers of hours students received instruction in both reading and math and academic achievement. Student Participation by 21st CCLC Site and Grade Of the 294 participants for whom data were collected, 64% were male and 36% were female. Over half of the students (55%) attended a program in one of the 1034 elementary schools. Thirty-eight percent (38%) attended one of the five middle schools and 6% attended one of the two participating high school 21st CCLC programs. Elementary school participants attended a mean of 26.61 (range = 85, SD = 23.60) days in the program, receiving a mean of 7.62 (range = 67, SD = 11.75) hours of reading-specific instruction and a mean of 4.39 (range = 35, SD = 7.56) hours of math-specific instruction. Middle school participants attended the 21st CCLC a mean of 19.39 (range = 91, SD = 17.08) days, receiving a mean of 4.16 (range = 71.5, SD = 10.71) hours of reading-specific instruction and 5.51 (range = 46.5, SD = 8.25) hours of math-specific instruction. High school 21st CCLC participants reported a mean of 49.80 (range = 69, SD 25.57) days attendance, .05 (range = 1, SD = .22) hours of reading instruction and 4.2 (range = 25, SD = 8.74) hours of math instruction (see Table 1). Skoglund Middle and Southwest High School reported zero hours of reading and math instruction during the 2007-2008 reporting period. Students who attended a high school program had higher mean days of participation than students who attended one of the elementary or middle schools 21st CCLC. High school participants attended a mean of 49.80 days in the 21st CCLC (range = 69, SD = 25.57). Middle school students attended a mean of 25.51 days (range = 92, SD = 22.58). Elementary school participants attended the 21st CCLC a mean of 26.61 days (range = 85, SD = 23.60). Elementary-age students between kindergarten and the fifth grade received a mean of 7.62 hours of reading instruction (range = 67, SD = 11.75) and a mean of 4.39 hours of math instruction (range = 35, SD = 7.56). Secondary-age students between grades 6 and 8 received a mean of 5.7 hours of reading instruction35 Table 1 Demographics by 21st CCLC Site 21st CCLC Numbers of Students Gender F M Mean Days of Participation Mean Hours of Instruction Reading Mean Hours of Instruction Math Elementary (k - 5) Belle Vista Elem 1 0 1 21.00 (range = .00, SD = 00) 8.00 (range = .00, SD = .00) 7.00 (range = .00, SD = .00) Centennial Elem 9 4 5 60.44 (range = 77, SD = 27.19) 3.44 (range = 31, SD = 10.33) 9.67 (range = 35, SD = 12.37) Garnet M 34 11 23 16.09 (range = 56, SD = 14.15) 11.59 (range = 51.5, SD = 14.16) 1.97 (range = 20, SD = 4.94) Haskin Elem 15 5 10 11.00 (range = 31, SD = 10.48) 4.27 (range = 19, SD = 7.20) 0.00 (range = .00, SD = .00) JB Elem 4 2 2 41.25 (range = 39, SD = 17.33) 14.25 (range = 14, SD = 6.40) 14.75 (range = 14, SD = 6.13) Kemper Elem 23 9 14 23.83 (range = 70, SD = 20.58) 4.85 (range = 23, SD = 7.04) 5.33 (range = 25, SD = 7.86) Lincoln Elem 32 17 15 33.44 (range = 78, SD = 21.83) 6.84 (range = 42, SD = 11.51) 4.17 (range = 31.5, SD = 8.42) Madison Elem 14 8 6 39.50 (range = 85, SD = 21.21) 3.21 (range = 20, SD = 6.75) 3.79 (range = 20, SD = 6.80) Manaugh Elem 14 1 13 21.21 (range = 85, SD = 32.09) 8.07 (range = 67, SD = 19.90) 0.00 (range = .00, SD = .00) Mesa Elem 16 3 13 27.00 (range = 49., SD = 12.63) 10.71 (range = 19.5, SD = 6.62) 10.59 (range = 19.5, SD = 6.67) Subtotals 162 60 102 26.61 (range = 85, SD = 23.60) 7.62 (range = 67, SD = 11.75) 4.39 (range = 35, SD = 7.56) 36 Table 1 (Continued) 21st CCLC Numbers of Students Gender F M Mean Days of Participation Mean Hours of Instruction Reading Mean Hours of Instruction Math Middle (6 - 8) Cortez 26 8 18 15.73 (range = 51, SD = 12.40) 6.50 (range = 19, SD = 6.50) 6.46 (range = 19, SD = 6.74) Delta 41 21 20 13.63 (range = 48, SD = 13.18) 0.72 (range = 9, SD = 1.91) 4.82 (range = 29, SD = 7.43) JE 30 4 26 25.13 (range = 91, SD = 21.65) 5.35 (range = 39, SD = 10.36) 7.60 (range = 46.5, SD = 13.53) Skoglund 7 4 3 27.00 (range = 49, SD = 17.15) 0.00 (range = .00, SD = .00) 0.00 (range = .00, SD = .00) Trinidad Subtotals 9 113 2 7 7 39 74 34.78 (range = 52, SD = 17.33) 19.39 (range = 91, SD = 17.08) 11.61 (range = 71.5, SD = 24.97) 4.16 (range = 71.5, SD = 10.71) 2.50 (range = 16.5, SD = 5.61) 5.51 (range = 46.5, SD = 8.25) High (9 - 12) Center 9 4 5 27.00 (range = 40, SD = 15.66) 0.11 (range = 1, SD = .33) 9.33 (range = 25, SD = 11.30) Southwest Subtotals 10 19 4 6 8 11 70.00 (range = 26., SD = 13.70) 49.80 (range = 69., SD = 25.57) 0.00 (range = .00, SD = .00) .05 (range = 1., SD = .22) 0.00 (range = .00, SD = .00) 4.2 (range = 25., SD = 8.74) 37 (range = 51.5, SD = 10.70) and a mean of 4.75 hours of math instruction (range = 46.5, SD = 8.25). Study participants between grades 9 and 12 received a mean of .05 hours of reading instruction (range = 1, SD = .223) and 4.2 hours of math instruction (range = 25, SD = 8.74) (see Table 2). Data Analysis Research Question 1: What is the correlation in Colorado and New Mexico between the number of days students with disabilities spend in 21st CCLC and their regular classroom teachers' perceptions of student progress toward reading and math proficiency on state standards? A Spearman rank-order correlation test was run to determine significant correlations between the numbers of days students with disabilities attended a 21st CCLC and each of the following: (a) reading grades, (b) teacher rating rubrics for reading, and (c) the PPICS teacher surveys for reading. Results from the Spearman rank-order correlation test indicated that no correlation reached significance at the .05 level between the number of days a student attended one of the 21st CCLC and students' reading grades (p<.05). No correlation reached significance at the .05 level between the numbers of days students attended a 21st CCLC and teacher rating rubric scores for reading. The Spearman rank-order test indicated that significance was not reached at the .05 level between 21st CCLC days of attendance and the PPICS teacher surveys for reading progress (p<.05) (see Table 3). A Spearman rank-order test was also run to determine significant correlations between the numbers of days students with disabilities attended a 21st CCLC and (a) math 38 Table 2 Demographics by Grade Level Grade Numbers of Participating Students by Grade Gender F M Mean Days Participation in 21st CCLC Mean Hours Participation Reading Mean Hours Participation Math K 10 5 5 23.00 (range = 40, SD = 28.28) .00 (range = .00, SD = .00) .00 (range = .00, SD = .00) 1 14 2 12 20.79 (range = 48, SD = 16.47) 4.82 (range = 15, SD = 5.76) 3.14 (range = 15, SD = 5.45) 2 23 10 13 33.48 (range = 79, SD = 27.44) 12.11 (range = 51.50, SD = 14.73) 4.79 (range = 23.60, SD = 7.64) 3 27 7 20 30.96 (range = 30.96, SD = 22.72) 13.17 (range = 42, SD = 12.90) 6.57 (range = 21, SD = 8.52) 4 46 24 22 23.72 (range = 85, SD = 22.67) 5.01 (range = 39, SD = 9.13) 2.60 (range = 16.50, SD = 4.58) 5 41 11 30 25.32 (range = 82, SD = 24.82) 5.72 (range = 67, SD = 11.95) 5.39 (range = 35, SD = 9.75) 6 37 13 24 18.81 (range = 70, SD = 14.83) 3.77 (range = 31.50, SD = 7.09) 7.92 (range = 42, SD = 9.89) 7 38 8 30 17.87 (range = 70, SD = 15.60) 5.67 (range = 71.50, SD = 13.12) 4.25 (range = 46.50, SD = 8.60) 8 38 18 20 21.47 (range = 91, SD = 20.45) 3.04 (range = 39, SD = 7.83) 4.43 (range = 33, SD = 8.66) 9 3 1 2 54.00 (range = 57, SD = 28.51) 0.33 (range = 1, SD = .577) 2.00 (range = 6, SD = 3.46) 39 Table 2 (Continued) Grade Numbers of Participating Students by Grade Gender F M Mean Days Participation in 21st CCLC Mean Hours Participation Reading Mean Hours Participation Math 10 4 1 3 30.25 (range = 31, SD = 15.09) 0.00 (range = .00, SD = .00) 14.00 (range = 25, SD = 12.94) 11 8 4 4 47.38 (range = 69, SD = 29.66) 0.00 (range = .00, SD = .00) 0.00 (range = .00, SD = .00) 12 5 3 2 66.80 (range = 29. SD = 14.84) 0.08 (range = .00, SD = .00) 4.40 (range = 22, SD = 9.84) Totals 294 107 187 25.38 (range = 92., SD = 22.63) 5.73 (range = 71.50., SD = 10.77) 4.82 (range = 46.50., SD = 8.29) 40 Table 3 Spearman Rank-Order Correlations Between Number of Days of Attendance and Reading Grades, Teacher Ratings, and PPICS Teacher Surveys in Reading Data Collection Tools n Correlation Significance p<.05 PPICS Survey 145 .064 .447 Teacher Rating 199 .047 .512 Reading Grades 170 .064 .410 *Correlation is significant at the 0.05 level grades, (b) teacher rating rubrics for math, and (c) the PPICS teacher surveys for math. The Spearman rank-order test showed no significance at the .05 level between the numbers of days a student attended one of the 21st CCLC and their regular classroom math grades (p<.05). Secondly, no significance at the .05 level was noted in the Spearman rank-order test between days of attendance and teacher rating scores in math (p<.05). However, there was significance at the .05 level between days of 21st CCLC participation and responses from the PPICS teacher survey regarding math (p<.05) (see Table 4). Table 4 Spearman Rank-Order Correlations Between Number of Days of Attendance and Math Grades, Teacher Ratings, and PPICS Teacher Surveys in Math Data Collection Tools n Correlation Significance p<.05 PPICS Survey 146 .171 *.039 Teacher Rating 198 -.078 .272 Math Grades 206 .029 .675 *Correlation is significant at the 0.05 level 41 Research Question 2: What is the correlation in two states, Colorado and New Mexico, between the number of hours students with disabilities spend in 21st CCLC receiving reading and math instruction and their regular classroom teachers' perceptions of student progress toward reading and math proficiency on state standards? The Spearman rank-order correlation test demonstrated a significance at the .01 level (p<.01) between the numbers of hours of reading instruction a student received and their regular day teachers' responses on the PPICS survey. The Spearman rank-order test yielded no significance at the .05 level between students' hours of reading instruction in a 21st CCLC and positive responses on the teacher rating rubrics in reading (p<.05). There was no significance at the .05 level between hours of reading instruction received and students' regular classroom reading grades (p<.05) (see Table 5). Regarding math achievement, the Spearman rank-order test demonstrated a significance at the .05 level between the number of hours of math instruction and math academic performance as reported through the teacher PPICS survey responses (p<.05). Table 5 Spearman Rank-Order Correlations Between Number of Hours of Reading Instruction and Reading Grades, Teacher Ratings, and PPICS Teacher Surveys in Reading Data Collection Tools n Correlation Significance p<.01) PPICS Survey 145 .266 **.001 Teacher Rating 191 .062 .397 Reading Grades 164 -.037 .634 *Correlation is significant at the 0.05 level ** Correlation is significant at the 0.01 level 42 There were no significance at the .05 level for hours of math instruction received in a 21st CCLC and math performance concerning the teacher rating rubrics, or student math grades (p<.05)(see Table 6). Table 6 Spearman Rank-Order Correlations Between Number of Hours of Math Instruction and Math Grades, Teacher Ratings, and PPICS Teacher Surveys in Math Data Collection Tools n Correlation Significance p<.05) PPICS Survey 146 .170 *.040 Teacher Rating 190 .074 .310 Math Grades 202 .068 .337 *Correlation is significant at the 0.05 levelCHAPTER IV DISCUSSION Review of the Problem The 21st Century Community Learning Centers (21st CCLC) were initially designed as safe places for low-income students to go after the regular school day ended (Halpern, 1999). The passing of The Improving America's Schools Act of 1994 ushered in dramatic increases in federal funding for 21st CCLC with a mandatory program emphasis on academic enrichment services (McCallion, 2003). The focus on academic priorities became more evident through the enactment of the No Child Left Behind Act of 2001 (NCLB). NCLB placed strict sanctions on schools, districts, and states that could not demonstrate academic proficiency in reading and math for all students, including a range of identified subgroups of students. One such subgroup was students with disabilities (No Child Left Behind Act of 2001: Academic Assessment, 2008). Title I schools function under high stakes for continuous academic improvement. In order for a 21st CCLC to be funded, it must demonstrate that one of its absolute program priorities (i.e., provided services) is academic enrichment activities. The statutorily mandated academic programming requirements for 21st CCLC paired with high stakes for Title I schools have created a situation where the academic services provided in 21st CCLC must supplement the goals and benchmarks for the students who attend the Title I schools44 whom they serve (No Child Left Behind Act of 2001: Local Competitive Grant Program, 2008). In 2005, Learning Point Associates was funded by the U.S. Department of Education to create a national evaluation of the 21st CCLC. The evaluation established the method by which all 21st CCLC data would be collected (Profile and Performance Information Collection System [PPICS]) and for whom data, including academic proficiency data, would be collected (Mitchell, Naftzger, Margolin, & Kaufman, 2005). Specifically, only students who attended a 21st CCLC for 30 days or more would have academic proficiency data collected on them. The identification of "30 days or more" as an indicator for data collection implies that there should be correlations between the amount of time students spend in a 21st CCLC and their progress toward program goals, especially academic program goals. The PPICS does not collect academic proficiency data for students with disabilities regardless of the number of days they attend. Beyond Learning Point Associate's national evaluation, there are no studies that speak to 21st CCLCs' impact on the academic progress of students with disabilities in their regular classrooms. This study was designed to determine if there were correlations between the numbers of days students with disabilities attended a 21st CCLC and the numbers of hours of reading and math instruction they received and their regular classroom teachers' perception of their academic progress in reading and math. Similarities and Differences Among Data Collection Sites This study examines data taken from multiple 21st CCLC from Colorado and New Mexico. In order to conduct more complete analysis on the data, it is important to 45 understand the similarities and differences between the 21st CCLC and how they may have impacted the study results. The following will explain how the instructional services in the 21st CCLC for this study were similarly intended to be delivered as described in the grant proposals written by Educational Research and Training Corporation (ERTC). Subsequently, program differences will be discussed as they relate to the results of this study. Lastly, in light of those program differences, an explanation of the similarities of the format in which the 21st CCLC reported student academic achievement will be given. All 17 grant proposals from which this study's data were collected were written by ERTC and subsequently evaluated by them. As such, there were comparable program elements regarding what data were collected in reference to the academic services provided in the 21st CCLC. As per each grant proposal, the academic services were to be completed under the direct administration of the students' regular day teachers using student success plans. In a student success plan, a student's regular day teacher identified state reading and math standards where the student in question had demonstrated deficiencies. Then, specific support activities and corresponding measureable goals were designed by the regular day teacher for implementation in the 21st CCLC. Depending on the teachers' purview of the student's academic needs, services were delivered in at least one of three types of instructional service. The three types of potential reading and math instruction were one-on-one tutoring, small group mini-courses and homework help. Tutoring services were delivered by either a paraprofessional, or a certified teacher facilitating a student's completion of any 21st CCLC supplemental activity as designed by the regular day teacher. Small 46 group mini-courses were designed through the cooperation of the regular day teacher and 21st CCLC staff. They consisted of 2 to 4 week scope and sequence lessons taught by paraprofessionals or certified teachers. Lastly, assistance with homework was offered to help 21st CCLC students complete unfinished classroom work from their regular school day. These services were also either provided by paraprofessionals or certified teachers. Even though the 21st CCLC services were defined similarly in the grant proposals as written by ERTC, it cannot be said that the educational reading and math services were the same. Each of the 17 sites employed its own staff, so the consistency and quality of the instructors likely varied across the sites. Also, each program had the flexibility to determine the days of the week and the time of the year that the 21st CCLC would be offered (e.g., summer programs, weekend programs, or before school). Since students were able to attend the 21st CCLC voluntarily, the regularity of when they received academic instruction in one of the programs may have been different. Another important difference of note is that although ERTC defined the parameters of what constituted tutoring services, mini-courses, and homework help, each site ultimately developed these activities individually. Some differences could be attributed to district- or school-adopted teaching strategies, or adopted educational programs. Although there were differences in the manner in which the study sites provided reading and math services to students, the format in which data were collected and reported by each site was the same. ERTC served as the program evaluator for the 17 21st CCLC programs from which data for this study were collected. Each program site 47 was required to report student academic progress in at least the following three reporting formats: (a) the PPICS teacher surveys regarding changes in student academic behavior in reading and math; (b) a teacher rating rubric based on academic gains with regards to state standards in reading and math; and (c) student grades converted to a rating scale rubric. This is important to note because these reporting formats were the data collection tools by which this study assembled the regular classroom teachers' perception of student academic proficiency as it related to their participation in a 21st CCLC. Though there were discrepancies in the academic services provided in the participating 21st CCLC, the manner in which student academic proficiency was reported was the same. Discussion of Study Results For students with disabilities attending 21st CCLC, academic performance data have not been reported. Although there has been prior research focused on analyzing the impact of after-school programming on student academic achievement, there have not been any investigations to determine the impact of 21st CCLC on the academic achievement of students with disabilities (Brooks et al., 1995; Dynarski et al., 2004; Fashola, 1998). Results from this study found that students with disabilities do indeed participate in academic enrichment activities in 21st CCLC (i.e., 294 students in 17 sites in New Mexico and Colorado). However, academic performance data may or may not be reported for students with disabilities who are participating in 21st CCLC. However, more in-depth academic data collection techniques than those currently utilized should be investigated regarding students with disabilities and the impact of the 21st CCLC on their 48 academic performance in regular classrooms. These data collection techniques need to first take into consideration the participation of students with disabilities in 21st CCLC and collect academic data in at least the same manner that academic data are collected for all other participating students. Data should be collected regarding the amount of academic instruction provided in the 21st CCLC and the type of instructional service provided. Research Question 1 What is the correlation in Colorado and New Mexico between the number of days students with disabilities spend in 21st CCLC and their regular classroom teachers' perceptions of student progress toward reading and math proficiency on state standards? The 21st CCLC program was reauthorized through the No Child Left Behind Act of 2001. NCLB (2001) directs that 21st CCLC focus on academic activities in order to assist the schools in achieving academic benchmarks and goals (No Child Left Behind Act: Purpose; Definitions, 2008). In 2005, the U.S. Department of Education funded Learning Point Associates to conduct the first-ever national evaluation of the 21st CCLC legislation since it was reauthorized and eventually funded as a state administered program (Mitchell et al., 2005). All data for this national evaluation were collected through the PPICS. This study followed the protocol of the 1999 Matematica Policy Research, Inc., evaluation which established the definition of regular attendance in a 21st CCLC (i.e., 30 days or more)(Dynarski et al., 2004). Due to the fact that all of the data were collected through the PPICS, the Learning Point Associates evaluation also instituted the data reporting parameters. The national evaluation implemented the parameter of exclusively 49 reporting academic performance data only for those students in regular attendance in a 21st CCLC (Mitchell et al., 2005). It did not report any academic performance data for students with disabilities. Subsequent national evaluations have maintained the same format and parameters of not collecting academic performance data for participating students with disabilities and only collecting academic performance data for those students in regular attendance in a 21st CCLC (Naftzger et al., 2006). The results from this study indicated no significance (p<.05) between the numbers of days students with a disabilities attended one of the 21st CCLC and their regular day teacher's perception of academic achievement in reading. This was true for all forms of data collected (i.e., the PPICS teacher surveys, teacher rating scores, and student grades). However, results from the study did demonstrate significance (p<.05) between the numbers of days that students attended a 21st CCLC and a positive PPICS survey response from students' teachers regarding math. Because significance (p<.05) was seen in math, but not reading, these results demonstrate that collecting academic performance data for students with disabilities only by days of participation in 21st CCLC ignores various programmatic factors that may have an impact on effectively evaluating those programs' success. Due to the fact that 21st CCLC are out-of-school-time programs, student participation, including students with disabilities who participated in this study, is voluntary. This factor warrants further explanation in discussing results from this study for Research Question 1. Students may have regularly attended a 21st CCLC (i.e., 30 days or more), but they may have chosen not to participate in the academic portions of 50 the program in favor of nonacademic program activities. Also, as a result of students participating voluntarily in the 21st CCLC, there were likely gaps in the consistency of instructional services provided. For example, even though the regular classroom teachers designed success plans for all participating students, they may have only attended the programs sporadically and may have only attended for a portion of the 21st CCLC programming day. The portion of the program day that the student attended may not have included academic instruction. Therefore, consistent and uniform instruction may not have occurred, resulting in the absence of significant correlations in the data with regards to days of participation and perceived academic progress in their regular classroom. As discussed in Chapter I of this study, the 21st CCLC are structured as supplemental academic programs to the lowest performing Title I schools (No Child Left Behind Act of 2001: Local Competitive Grant Program, 2008). As a supplemental program to Title I schools, 21st CCLC are being called upon to assist in achieving each school's required benchmarks for proficiency for all students, including students with disabilities (No Child Left Behind Act of 2001: State Plans, 2008). Research has demonstrated that after-school programs may have an impact on student academic performance in the regular day classroom. One example is the LA's BEST study whose results suggested that students who participated in the after-school program for 2 years had measured increases in their grade point averages from their first year of participation to the second (Brooks, Mojica, & Land, 1995). The Fashola report (1998) went so far as to identify successful after-school designs and practices for achieving academic goals for 51 participating at risk students (Fashola, 1998). The problem with these two studies is that neither collected data specifically for students with disabilities. The academic performance data specifically for students with disabilities need to be included in 21st CCLC evaluations so that effective academic enrichment activities can be created and implemented. The results from this study suggest that the current 21st CCLC structure for reporting academic performance data by regular attendance (i.e., 30 days or more) is insufficiently comprehensive for determining the impact of the programs on the academic performance of students with disabilities. Research Question 2 What is the correlation in two states, Colorado and New Mexico, between the number of hours students with disabilities spend in 21st CCLC receiving reading and math instruction and their regular classroom teachers' perceptions of student progress toward reading and math proficiency on state standards? As discussed above, the 21st CCLC are required to focus on academic activities and their program purpose have been aligned as a supplemental academic enrichment to Title I schools in order to assist them in meeting academic benchmarks (No Child Left Behind Act of 2001: Purpose; Definitions, 2008). Emphasizing the role as an academic supplement to students' regular school day, the funding for 21st CCLC has increased dramatically (United States Department of Education, 2007). The Fashola study (1998) supports academic emphasis and suggests that successful after-school programs should stress the academic component of their enrichment activities. In response to this, Research Question 2 was designed to focus only on the academic data (i.e., reading and math) and to expand upon the reporting 52 parameters (i.e., 30 days or more) created by the Learning Point Associates studies (Mitchell et al., 2005; Naftzger et al., 2006). This study found that significance exists at the .01 level for hours of reading instruction students received and their regular day teachers' responses on the PPICS teacher survey (p<.01). Significance was also found at the .05 level for hours of math instruction and teacher responses on the PPICS teacher survey (p<.05). In contrast to the findings for Research Question 1, these findings propose that through more in-depth data collection procedures than those currently used in the PPICS, academic performance data can be obtained for students with disabilities who participate in 21st CCLC. National evaluations of the 21st CCLC report data as collected through the PPICS and academic data are limited to the PPICS surveys and state assessment scores. As previously discussed, academic data for students with disabilities are not collected through the PPICS (Learning Point Associates, 2008). Data for the current study were taken from a convenience sample and state assessment scores were not available. However, teacher rating rubrics and grades were available for examination. The teacher ratings and grades are similar in that they demonstrate student academic performance based on classwork and classroom behavior as related to reading and math. Although it did not include data specifically for students with disabilities, the often cited LA's BEST study found that students who participated in its after-school programs for at least 3 months for 2 consecutive years showed marked increases in their general grade point averages (Brooks, Mojica, & Land, 1995). However, no results were noted in this 53 current study for hours of reading or math instruction and student achievement as reported through the teacher rating rubrics or student grades. It is unclear as to what factors contributed to significance (p<.05) between the PPICS teacher surveys for hours of both reading and math instruction and teachers' perception of student academic improvement, but not for either the teacher ratings or student grades and hours of program participation. These results do suggest the need for expanded methods of collecting academic performance data for students with disabilities as it relates to their participation in 21st CCLC. Study Limitations There are a number of limitations of this study that must be noted. This study utilized a convenience sample. As such, external validity is problematic in that study conclusions are not generalizable (Drew, Hardman, & Hosp, 2008). There were a number of factors that were not included in the data collection that may have had an influence on the study results. Student participation in the 21st CCLC was voluntary; no information was provided regarding how consistently students attended the program and the quality of instruction they received. In other words, it was unknown whether the days of student participation were consecutive, or if the hours of reading and math instruction were linked to the state standards the students were studying in their regular classroom. While there were similarities in the overall designs of the 21st CCLC services because ERTC wrote the proposals for each cite. Each cite hired its own staff and developed and delivered its own academic services (i.e., tutoring services, math and reading mini-courses, and homework help services). As a result of each 21st CCLC implementing the 54 program design independently from one another, the quality and intensity of the instructional services provided were also not exactly the same from one 21st CCLC to the next. Another limitation of the study regards the absence of information regarding the different classifications of disabilities of the students from whom data was collected. Although all of the students whose data were collected for this study were identified as receiving special education services during their regular school day, it was unknown which disability they had. Certain disabilities could have influenced students' ability to progress academically in their regular classroom to an extent that would have been recognized by their regular classroom teachers. Limitations of this study are extended to the type of data that was collected. Considering that all 17 21st CCLC utilized the same type of reporting forms (i.e., teacher ratings, grades, and PPICS surveys), the level of data collected were ordinal. This disallows the use of more sensitive statistical analysis, thus providing less stable relationships between the number of days of student participation in a 21st CCLC, the number of hours of reading and math instruction, and students' academic achievement in the regular school day (Drew, Hardman, & Hosp, 2008). Implications for Practice Under current federal education law, Title I schools are required to meet strict standardized academic progress measurements for all students, including defined subgroups of students. One of those identified subgroups for which these schools are required to demonstrate academic progress is students with disabilities. The law requires 55 that 21st CCLC serve students that attend Title I, Part A schools. They are also required to serve the lowest academically performing Title I schools (No Child Left Behind Act of 2001: Local Competitive Grant Program, 2008). A second requirement of those 21st CCLC is that their primary focus be to provide academic enrichment in support of the Title I school whose students they serve (No Child Left Behind Act of 2001: Purposes; Definitions, 2008). The legally mandated cooperation of 21st CCLC and the Title I schools that they serve demonstrates the importance of identifying successful instructional strategies and program implementation in the 21st CCLC that will ensure academic success for the students that attend them. This would also be true for the students with disabilities that attend 21st CCLC. There have been a number of studies, including a continuous national evaluation regarding 21st CCLC and their impact on student progress towards academic goals (Brooks, Mojica, & Land, 1995; Dynarski et al., 2004; Fashola, 1998; Naftzger et al., 2006). However, no studies, including these cited studies, provide data regarding the impact of the 21st CCLC on the academic progress for students with disabilities in their regular day classroom. This study revealed that students with disabilities indeed attend 21st CCLC and participate in the program-required academic reading and math services offered in them. Therefore, a future challenge for policy makers and evaluators of 21st CCLC programs is to develop analyses that specifically include measurements for students with disabilities. Future evaluation procedures need to include better measurements for students with disabilities in the state standards in the areas of math and reading as they relate to students' IEPs. 56 All 21st CCLC must collect and submit program data, including academic achievement data, through PPICS (U.S. Department of Education Office, 2004). The PPICS data system requires that 21st CCLC collect and evaluate academic progress only for those students that attend the program for 30 days or more (i.e., regular attendees). While 21st CCLC are required to contain academic enrichment activities, they are not limited to offering only those activities. As a result, most 21st CCLC offer companion activities to their academic instruction during their program time. Examples of these might be sports, arts, or crafts. Because student participation in 21st CCLC is largely voluntary, there may be no individual program requirements that students participate in the academic enrichment activities offered. Evaluation practices for 21st CCLC need to account for the quantity and quality of core standard academic educational enrichment activities in reading and math before determinations of their effectiveness can be made. This would be especially true for students with disabilities who may have learning issues that limit their academic progress, or may have an individualized education program (IEP) that would draw evaluative emphasis away from state standard focused academic progress. As shown in this study, evaluation practices and data collection techniques need to be more sensitive for students with disabilities that attend 21st CCLC before academic program effectiveness can be determined for them. The results from this study should be examined by federal policy makers and the national evaluators of the 21st CCLC. Evaluation mechanisms need to be included in the current evaluation procedures that can account specifically for the academic impact that 21st CCLC have on students with disabilities that participate in them. Evaluation and 57 data collection systems for the 21st CCLC need to be more sensitive in order to address factors that are unique to students with disabilities so that more conclusive decisions can be made for them and their participation in 21st CCLC. Results from this study should be examined by state and federal policy makers as they suggest that academic data collection practices for 21st CCLC are problematic, not only for students with disabilities, but for students in general that participate in them. As discussed earlier, all data for 21st CCLC are collected through PPICS, which allows states to determine what types of academic data are collected for purposes of monitoring, evaluation, and technical assistance (Fortune, 2006). This practice is limited to collecting the state academic data exclusively for students that are regular attendees (Learning Point Associates, 2008). Current practices do not account for the relationship of 21st CCLC student participation and their academic proficiency in their regular day classroom. Student participation in 21st CCLC is voluntary and current academic data collection practices do not discern between low academic performing students' participation and their high academically performing 21st CCLC participant counterparts. An example of this problem was suggested through the results of this study, specifically for students with disabilities and their potential under-identification for participation in 21st CCLC. Nationally, students with disabilities make up approximately 13% of the overall student population as opposed to less than 1% (.85%) of all students that participated in the 21st CCLC from which data were extracted for this study (National Center for Education Statistics, 2012). Additionally, currently there is no program data collection practice to determine the quantity, consistency, or quality of the academic services that participating 58 students receive in 21st CCLC. This makes evaluating the effectiveness of the services provided in those programs troublesome. Therefore, the continuous improvement of academic portions of the programs is difficult. These problematic 21st CCLC academic data collection practices should be evaluated by state and federal policy makers in order to make changes that will result in the collection of more useful data for more effective and impactful program design and implementation. Implications for Future Research There has been a recent growing political trend and subsequent legal mandate for stricter accountability measures for educational programs, including 21st CCLC. This has resulted in an emphasis on data collection and program evaluation. However, for students with disabilities and the impact their participation in 21st CCLC has on academic achievement, little or no data have been collected. Therefore, conclusions regarding successful 21st CCLC academic enrichment strategies and program implementation for students with disabilities cannot be made. In that regard, future research needs to be done that is focused specifically on the impact that 21st CCLC have on the academic progress towards proficiency of students with disabilities in reading and math. The results from this study suggest a relationship between the hours of reading and math instruction that a student with a disability received during participation in a 21st CCLC and their regular day teachers' perception of academic improvement. However, because the data for this study originated from a convenience sample, it lacked important informational components. These important information factors included, but were not limited to, demographical issues, teacher quality, teaching methods implemented, and 59 unique student background information. Also, this study aggregated data from 21st CCLC at the elementary, middle school, and high school levels. As academic expectations are different between these levels of school (e.g., grades, scheduling, and end of level tests), future studies will need to disaggregate data for these grades to account for these distinctions to more precisely report study results. These factors may have impacted this study's results. More in-depth research needs to be undertaken that accounts for these factors and their impact on study results. Also, future research needs to be conducted where methodology, design, and statistical analysis can be conducted that allow for more generalizable conclusions. Research that yields generalizable conclusions will provide important data to administrators, teachers, and staff so that they can design and implement 21st CCLC that provide for more effective instructional services and strategies for students with disabilities. Summary The purpose of this study was to (a) determine if there were correlations between the numbers of days that students with disabilities attended a 21st CCLC and their regular day classroom teachers' perception of reading and math progress towards proficiency; and (b) determine if there were correlations between the numbers of hours that students with disabilities received reading and math instruction in a 21st CCLC and their regular classroom teachers' perception of reading and math progress towards proficiency. Data for this study were gathered from a convenience sample of 294 students that attended one of 17 21st CCLC in either Colorado or New Mexico during the 2007-2008 program year. The data were collected by Educational Research and Training Corporation as part of 60 each site's program evaluation, which was also conducted by them. All students for whom data were collected were identified as receiving special education services during the regular school day, having an IEP, and as having spent the majority of their school day in a regular classroom setting. Reading and math academic data on this study's participants were collected through grades, a teacher rating rubric, and a federally mandated teacher survey. The results for this study suggest that there is a relationship between the numbers of hours of academic services in both reading and math that students with disabilities received in a 21st CCLC and their regular classroom teachers' perception of progress towards proficiency as determined by the PPICS survey. With the exception of days of participation in a 21st CCLC and teachers' responses on the PPICS survey for math, no significance (p<.05) was found regarding the numbers of days study participants attended and their progress towards proficiency. Neither in reading nor math could significance be found for teacher rating rubric scores or for students' grades. However, considering that the information collected for this study was taken from a convenience sample, there were limitations to the conclusions that could be made. Important pieces of information were missing from the data that may have impacted the results. Results from this study suggest that current 21st CCLC evaluation procedures are insufficient to account for the academic impact that 21st CCLC have specifically on students with disabilities that participate in them. Evaluation and data collection systems for the 21st CCLC should be more sensitive in order to account for factors that are unique to students with disabilities so that more conclusive decisions can be made for them and 61 their participation in 21st CCLC. These results should be examined by federal policy makers and national 21st CCLC evaluators so that valuable research continues and expands, advancing programming and instructional strategies for participating students with disabilities.REFERENCES Brooks, P. E., Mojica, C. M., & Land, R. E. (1995). Final evaluation report: Longitudinal study of LA's best after-school education and enrichment program, 1992-1994. Retrieved from University of California, Center for the Study of Evaluation & Information Studies website: http://www.cse.ucla.edu/products/misc/cse_brooks_long.pdf. Colman, P. (1995). Rosie the riveter: Women working on the home front in World War II. New York: Crown Publishers. Drew, C.F., Hardman, M.L., & Hosp, J.L. (2008). Designing and conducting research in education. Thousand Oaks: Sage. Dynarski, M., James-Burdumy, S., Moore, M., Rosenberg, L., Deke, J., & Mansfield, W. (2004). When schools stay open late: The national evaluation of the 21st century community learning centers program (Contract No. ED-99CO-0134). Retrieved from U.S. Department of Education; Office of the Under Secretary website: http://www2.ed.gov/rschstat/eval/other/learningcenters/clcreport2004.pdf. Examining the 21st century community learning centers program: Hearing before the Subcommittee on Early Childhood Youth and Families of the Committee on Education and the Workforce, House of Representatives, 106d Cong. 2 (2000). Fashola, O. S. (1998). Review of extended-day and after-school programs and their effectiveness (CRESPAR Report 24). Retrieved from Johns Hopkins University website: http://www.csos.jhu.edu/crespar/techReports/report24.pdf. Fortune, A. (Moderator). (2006, April). CCSSO-council of chief state school officers: Monitoring and evaluating SES and 21st CCLC program conference call (Conf. 5140036), Council of Chief State School Officers. Government Performance and Results Act of 1993, 31 U.S.C. § 1101 (1993). Halpern, R. (1999). After-school programs for low-income children: Promise and challenges. The Future of Children: When School Is Out, 9 (2), 82, 83. Retrieved from http://futureofchildren.org/futureofchildren/publications/docs/09_02_07.pdf. Ianni, F.A.J. (1989). The search for structure: A report on American youth. New York: The Free Press. 63 Improving America's Schools Act, 20 U.S.C. §§ 10904, 10905 (1994). Learning Point Associates. (2008). Profile and performance information collection system (PPICS). Retrieved from http://ppics.learningpt.org/ppicsnet/public/default.aspx. McCallion, G. (2003) Report for Congress: Received from the Congressional Research Service, Order Code: RL31240. 21st Century Community Learning Centers in P.L. 107-110: Background and Funding. Retrieved from http://assets.opencrs.com/rpts/RL32174_20031216.pdf. McLaughlin, M. J., Embler, S., & Nagle, K. (2004). Students with disabilities and accountability: The promise and the realities should there be alternatives? Report for the Center on Education Policy. Miller, K., Snow, D., & Lauer, P. (2004). Noteworthy perspectives: Out-of-school time programs for at-risk students. Mid-continent Research for Education and Learning. Retrieved from http://www.readingrockets.org/article/15031/. Mitchell, C.J., Naftzger, N., Margolin, J., & Kaufman, S. (2005). 21st century community learning centers (21st CCLC) analytic support for evaluation and program monitoring: An overview of the 21st CCLC program: 2003-04. Learning Point Associates. Naperville, Illinois. Retrieved from http://www.childcareresearch.org/childcare/resources/7498/pdf. Moore, K.A., Driscoll, A.K., & Lindberg, L.D. (1998). A statistical portrait of adolescent sex, contraception, and childbearing. Washington, DC: The National Campaign to Prevent Teen Pregnancy. Naftzger, N., Margolin, J., Kaufman, S., & Ali, A. (2006). 21st century community learning centers (21st CCLC) analytic support for evaluation and program monitoring: An overview of the 21st CCLC program: 2004-05. Learning Point Associates. Naperville, Illinois. 32, 37, 47. Retrieved from www2.ed.gov/programs/21stcclc/2006report.doc. National Center for Education Statistics. (2012). The condition of education: Children and youth with disabilities. Retrieved from http://nces.ed.gov/programs/coe/indicator_cwd.asp. No Child Left Behind Act of 2001: Academic Assessment, 20 U.S.C. § 6316 (2008). No Child Left Behind Act of 2001: Academic Assessment and Local Educational Agency, 20 U.S.C. § 1116 (2008). No Child Left Behind Act of 2001: Allotments to States, 20 U.S.C. § 4202 (2008). 64 No Child Left Behind Act of 2001: Local Activities, 20 U.S.C. § 4203 (2008). No Child Left Behind Act of 2001: Local Competitive Grant Program, 20 U.S.C. § 4204 (2008). No Child Left Behind Act of 2001: Purpose; Definitions, 20 U.S.C. § 4201(2008). No Child Left Behind Act of 2001: State Application, 20 U.S.C. § 4203 (2008). No Child Left Behind Act of 2001: State Plans, 20 U.S.C. § 6311 (2008). Non-Regulatory Guidance for 21st Century Community Learning Centers. (2003). Retrieved from http://www.ed.gov/programs/21stcclc/legislation.html. Office of Innovation and Improvement; Overview Information; High-Quality Supplemental Education Services and After-School Partnerships Demonstration; Notice Inviting Applications for New Awards for Fiscal Year (FY) 2008, 73 Fed. Reg. 38430-38435 (2008). Office of Management and Budget (2007). Government Performance and Results Act of 1993. Retrieved from http://www.whitehouse.gov/omb/mgmt-gpra/gplaw2m.html#h2. Olson, L. (2004). Special education in an era of standards count me in. Education Week, 23(17), 8. Retrieved from http://www.edweek.org/media/ew/qc/archives/QC04full.pdf. Pett, M.A., (1997). Nonparametric statistics for health care research statistics for small samples and unusual distributions. Thousand Oaks: Sage. Rose, E. (1999). A mother's job: The history of day care, 1890-1960. New York: Oxford University Press. Seppanen, P., DeVries, D., & Seligson, M. (1993). National study of before- and after-school programs. Washington, DC: Office of Policy and Planning, U.S. Department of Education. Retrieved from ERIC database. (ED356043). United States Department of Education. (2007). Budget history tables: State funding history tables by program. Retrieved from http://wdcrobcolp01.ed.gov/CFAPPS/CCLC/allstate_cclc1.cfm. U.S. Department of Education Office. (2004). Federal Register Online Notices. (FR Doc. E4-2198). Retrieved from Federal Register Online Notices via GPO Access: http://www.ed.gov/legislation/FedRegister/other/2004-3/091604b.pdf.