Over the last decade, a number of large-scale studies have examined whether attending a charter school can improve students’ performance on standardized tests and other outcomes, compared with attending a traditional public school. Among the key players in this field of research are the Center for Research on Education Outcomes (CREDO) at Stanford University, the public policy research institutes Mathematica and the RAND Corporation, and a number of academic researchers from various universities around the nation.
Before reviewing the main findings of this research, it is important to understand some basic facts about the methodologies of the most reputable studies in the field.
Studies that are considered methodologically most sound typically study charter school impacts in one of two ways:
It is also important to keep in mind that any research results on charter school impacts on students—from lottery as well as matching studies—can be generalized only to students (and families) who want to attend charters. By definition, charters are schools of choice, and the study samples are therefore limited to students and families who might choose a charter school instead of a traditional public school. This is particularly important to consider if charter applicants systematically differ from students and families who do not apply to charter schools in ways that could affect students’ academic performance. For more information on what is known about whether charters seem to attract more or less prepared students and families, see the Diversity and Inclusion and Families sections.
The main take-away from studies that have examined charter schools’ impacts on students’ performance on standardized tests is that these impacts vary widely across states, types of students, types of schools and over time. Moreover, the research is continuously evolving as more and better data become available.
Here, we summarize the current state of knowledge of charter schools’ impacts on student achievement. We summarize the field’s key findings, nationally and across states, student demographics, types of schools and over time. Our review focuses on the largest and most recent studies and selected local studies. It is not comprehensive, but it provides a general overview of charter education impacts in the literature.
Nationally, there is very little evidence that charter and traditional public schools differ meaningfully in their average impact on students’ standardized test performance. The most recent CREDO research (matching study) drew on data through 2010–11 from 27 states and found that charter schools on average had a small positive impact on students’ reading achievement but no differential impact on students’ math achievement. Note that CREDO counts Washington, D.C., as a state and also counts New York City as a “state" separately from New York State.2 A 2014 meta-analysis of the literature on charter school effects, by researchers with the Center on Reinventing Public Education — a research and analysis organization associated with the University of Washington Bothell that focuses on “innovative schools of choice” and that works to “develop, test, and support evidence-based solutions to create new possibilities for the parents, educators, and public officials who strive to improve America's schools”— reported no significant impacts on reading scores and small positive impacts on math scores.3
However, there are many different kinds of charter schools, many different types of students and wide variation in states’ charter school laws and hence significantly different regulatory contexts in which charter schools operate. For more information on these regulatory and operational variations, see the Charter School Operators section. Considering these variations, research findings become more nuanced and more meaningful:
In some states charter schools have had positive impacts on student learning, in other states they have had negative impacts, while in others charters have had no differential impact compared with traditional public schools. For example, the most recent CREDO research (matching study) used data through 2010–11 and reported that in 16 states, charter schools were associated with greater reading gains compared with traditional public schools. In eight states, they found negative impacts for charter schools, and in three states they found no differences between charter school and traditional public school students’ reading improvement, on average (the study included 27 states). The differences ranged from charter schools showing reading gains equivalent to 86 days more learning than in traditional public schools in Rhode Island and Tennessee, to reading gains equivalent to 108 days less learning than in traditional public schools in Nevada.4 For the full list of charter school impacts by state, see pages 52–53 of CREDO’s National Charter School Study 2013.
CREDO authors pointed out that the diversity of charter impacts in generally low-performing areas is particularly noteworthy. On the one hand, charter impacts were associated with the equivalent of 101 extra days of math learning in Washington, D.C. and 91 extra days of learning in New York City. In these cases, one can argue that charters improve educational opportunities for students. On the other hand, when charter schools’ impacts are negative in already low-performing states (for example, Nevada) one may argue that they are further limiting educational opportunities for students.
Lower-income and urban students are most likely to benefit from a charter education. A number of studies that focus on charter schools in large urban districts (for instance, New York City, Boston and Los Angeles) found positive impacts of charters on students’ standardized test score achievement. For example, Caroline Hoxby and colleagues’ evaluation of New York City charter schools (lottery study; nearly all New York City charter schools are lottery based) analyzed data through 2007–08 and reported positive impacts for charter schools on students’ achievement in English and math. The size of these effects increased as students spent more years in charter schools.5 Similarly, Atila Abdulkadiroglu and colleagues (lottery study) analyzed data through 2006–07 and reported positive impacts on English and math achievement among charter middle and high school students in their evaluation of Boston charter schools.6
These single-city studies are highly informative for charter school debates in those respective cities, but their findings are not easily generalizable beyond their specific locations and population. First, locations vary in their charter school laws and oversight, which in turn may differentially affect charters’ impacts. Second, as charter school students in large urban cities such as New York City, Boston and Los Angeles are predominantly African-American or Hispanic and living in poverty, these single-city studies cannot tell us whether the impacts are specific to low-income students, minority students, urban schools, a combination or none of these factors.
National studies, such as the 2013 27-state CREDO study and Mathematica’s 2011 evaluation of fifth through seventh graders in 36 schools across 15 states, can help disentangle some student subgroup and locale impacts.
CREDO (2013, matching study, including schools across 27 states) analyzed data through 2010–11 and found charter schools’ greatest impacts on the math and reading growth of low-income, minority students (low-income is measured as students’ eligibility for free or reduced-price lunch). For example, they reported math gains equivalent to 36 extra days of learning for African-American students living in poverty, compared with African-American students living in poverty and attending traditional public schools. The study reported no differential impacts for African-American charter school students who were not living in poverty. Similarly, CREDO 2013 found positive impacts for low-income Hispanic students, but negative impacts for Hispanic students who were not living in poverty. Impacts on low-income, minority students were especially pronounced in urban areas.
Mathematica’s evaluation of charter middle schools in 15 states (lottery study) examined data through 2007–08 and also found that both students’ background and the location of the school mattered for achievement outcomes. The study reported small positive impacts for low-income students (those eligible for free or reduced-price lunch) and somewhat larger negative impacts for higher-income students in reading and math. At the same time, the study found that regardless of income, urban charter schools were more likely to have positive impacts on students’ math achievement than nonurban charter schools, which tended to have negative impacts. After conducting further exploratory analyses, the authors noted that urban charter schools’ relative edge could be due to their urban comparison schools constituting lower-quality alternatives than the study’s suburban and rural comparison schools, especially for low-income students.7
One charter school’s impact can differ greatly from another’s. Some school characteristics have been found to relate to charter impacts. Every charter evaluation study has reported great variation among individual charter schools’ impacts and among different types of charter schools’ impacts. For example, CREDO 2013 (matching study) analyzed data through 2010–11 and found that the majority of the schools in their study (56 percent) had no greater or lesser impact on their students’ reading gains than did traditional public schools, but 25 percent of the schools in their study improved students’ reading over and above traditional public schools’ impact, while 19 percent diminished students’ reading gains compared with traditional public schools. The same study also found more positive impacts for charters that were elementary or middle schools, but no impacts for charter high schools.8
Similarly, Mathematica studied impacts of 22 charter management organizations (CMOs) (matching and lottery study). CMOs are nonprofit organizations that manage multiple charter schools. For more information about these organizations, see the Charter School Operators section. Examining data through 2010–11, Mathematica reported differences in impacts that ranged from the highest-impact CMO demonstrating three years’ worth of learning gains within a two-year period to the lowest-performing CMO showing as little as one year of learning gains in a two-year period. Overall, the study found positive impacts on students’ math and reading scores in 45 percent of the CMOs studied, but just as many CMOs had negative impacts in either math or reading.9
Studying impacts of a specific CMO, the Knowledge Is Power Program (KIPP) (matching and lottery study), Mathematica examined data through 2010-11 and found that while KIPP middle schools had overall positive impacts on student learning in reading and math, some KIPP middle schools had greater impacts than others.10
Such variation raises the question of what aspects of a charter school make it more or less likely to impact students’ learning compared with traditional public schools. While results pertaining to this question are correlational and hence preclude us from making direct and confident attributions about cause, the relationships are suggestive in ways that warrant further study. For example, positive charter school impacts have been correlated with:
For more information about the research on these and other practices, see the Innovation section.
There is comparatively little rigorous research comparing charter schools with traditional public schools on measures of academic outcomes other than standardized test scores, such as indicators of academic engagement and motivation, high school completion, college matriculation, graduation and so on. These tend to be long-term outcomes that cannot be measured until several years after students entered a charter school. Moreover, much of this information is not collected in the same comprehensive ways as standardized test scores.
However, a growing number of studies have looked at additional measures of academic engagement and success. Their results typically mirrored what they reported on charters’ impacts on standardized test scores.
For example, Mathematica researchers’ 15-state study of charter school fifth through seventh graders (lottery study)—a study that examined data through 2007–08 and reported no differential impacts for charters overall—reported no evidence that charter school students had better school attendance or grade promotion than their peers in traditional public schools.12
However, studying charter schools’ impacts in New York City with data through 2007–08, Caroline Hoxby and colleagues (lottery study) not only found positive charter impacts on math and reading scores, but also reported that the longer students stay in charter schools, the higher they scored on the New York State high school Regents examinations and the more likely they were to earn a New York State Regents diploma. Similarly, Joshua Angrist and colleagues’ 2013 lottery-based evaluation of Boston charter schools reported positive impacts for charter schools not only on students’ standardized test scores, but also on indicators of college preparedness, including SAT scores and students’ likelihood to enroll in a four-year college.13 Will Dobbie and Roland Fryer, too, found in their 2012 survey of former students that winning a lottery-based place at Promise Academy in Harlem increased students’ college enrollment rate and especially their likelihood to enroll at a four-year school.14
Evaluating charters in eight locations (matching study), Ron Zimmer, Brian Gill and colleagues reported positive impacts of charter school education on academic outcomes other than standardized test scores, using data through 2006–07. In a sample of charter middle school students in Florida and Chicago, students who transferred into a charter high school were more likely to graduate from high school and more likely to attend college than those charter middle school students who did not transfer into a charter high school.15
Mathematica researchers reported that students who were offered a lottery-based place at KIPP middle schools not only performed better on standardized math and reading tests than students who did not win a seat in a KIPP school, but also reported doing more homework and being more satisfied with their school. At the same time, the study found no difference between KIPP lottery winners and non-winners on such measures as academic engagement and effort and educational aspirations (all self-reported by students).16
Again, there is significant variation in the extent to which charter schools impact students’ academic engagement and success. Across six CMOs, Mathematica (matching study) found no overall impact on high school graduation, but substantial variation among CMOs, ranging from one increasing students’ probability of graduating by 23 percent to another that reduced students’ likelihood of graduating high school by 22 percent.
There is hardly any research so far that estimates how a charter school education shapes students’ future employment trajectories and income. One exception is a matching study by Kevin Booker and colleagues. Leveraging long-term student tracking data from Florida through 2006–07, this study matched charter high school students who had also attended charter middle schools with charter middle school students who had not attended charter high schools and found that in their early to mid-twenties, former charter high school students had significantly higher earnings.17
The mission of public education extends beyond academic learning to include the preparation of students to be engaged citizens. An important research question is therefore how successful charter schools are in developing students’ civic knowledge, skills and attitudes, in both the short and the long term.
Civic engagement outcomes are again more difficult to measure and track than students’ standardized test scores. Hence there is so far little research on this issue. In one early matching study, Jack Buckley and Mark Schneider reported that charter school students who were surveyed in 2003 reported more civic skills training and community participation but were no different in their endorsement of civil liberties than their peers in traditional public schools.18
Recent studies have not included civic engagement outcomes, even though a number of charter school networks focus explicitly on civic education.
The literature on charters’ impacts on behavioral and health-related outcomes is limited to small-scale studies that focus on specific charter schools, and their findings are mixed.
Will Dobbie and Roland Fryer analyzed data through 2012 and reported a decline in teenage pregnancy for females and a decline in incarceration rates among males who won a place at Promise Academy in Harlem, compared with their peers who did not win a lottery-based seat at the school. However, the study found impacts neither on behaviors such as alcohol or drug use nor on health outcomes. Information on these outcomes was collected through surveys with students.19
Mathematica’s evaluation of KIPP middle schools (lottery study) utilized data through 2010-11 and found that KIPP lottery winners were also more likely to say they argued with their parents, lost their temper, lied or gave teachers a hard time than students who applied to KIPP but did not win a place—results the authors suggest could reflect true differences in behavior or differences in students’ likelihood to honestly report such behavior. At the same time, Mathematica found no difference in how KIPP lottery winner parents and parents whose children did not win a lottery-based seat described their children’s behavior outside of school and behavioral problems.20
It is generally acknowledged that a lot more research, and much more data, is needed to understand if, how and when charter schools shape children’s academic and nonacademic outcomes.
To fully understand charter schools’ impact on all students’ achievement, it is important to consider whether the existence and expansion of charter schools affect academic and nonacademic outcomes of students in nearby traditional public schools.
Advocates have argued that charter schools carry the potential to increase competition in the education market and thus encourage traditional public schools to do more to improve student performance. Proponents have also often pointed out that charter school innovations could spread to traditional public schools, an issue that our section, Innovation, discusses in more detail. Skeptics have pointed out that the existence of charter schools could negatively impact the education and performance of students in nearby traditional public schools by taking financial resources away from those public schools and by attracting the most motivated and engaged families (“cream skimming”). For more information on these issues, see the Diversity and Inclusion and Finances sections.
Studies have measured charter schools’ “pressure” on traditional public schools by a) the proportion of public school students in a given district who are enrolled in charter schools, or b) the number of charter schools located within a certain radius of a traditional public school, or c) the percentage of students a school has lost to charter schools each year. These studies use longitudinal student test score data to examine whether traditional public school students’ performance changed with increasing pressure from charter schools.
The results of these studies are mixed and vary notably across location. Moreover, whenever impacts were found, they have typically been small.
Some studies have found no evidence that charter school competition affected the performance of students in nearby traditional public schools. For example, Ron Zimmer and Richard Buddin, using data through the 2001–02 school year, found no evidence that competition from charters impacted the test score performance of traditional public school students in California.21 Similarly, Robert Bifulco and Helen Ladd, also using data through the 2001–02 school year, found no evidence for such impacts in North Carolina.22 Ron Zimmer, Brian Gill and colleagues found no impact in seven of the eight states and districts they studied, using data through the 2007–08 school year.23 Analyzing spring 2002 data from a national student sample, Tomeka Davis, too, found no evidence that charter school competition affected students’ achievement in traditional public schools.24
Some studies have reported findings that showed positive impacts of charter school competition on the standardized test scores of students in nearby traditional public schools. For example, Kevin Booker and colleagues analyzed data from Texas through the 2003–04 school year and found that students’ performance in traditional public schools improved as charter schools came to their districts. They found particular improvement among low-income and African-American and Hispanic students in traditional public schools.25 Marcus Winters also found some evidence in New York City that traditional public school students’ scores improved slightly as charter school competition grew, analyzing data through the 2008–09 school year.26 Using data through the 2004–05 school year and a different statistical method from that of Robert Bifulco and Helen Ladd’s study cited above,27 Yusuke Jinnai found small positive impacts in North Carolina.28 Small positive impacts have also been reported for Florida.29
However, Yongmei Ni, using data through 2003–04 from Michigan, reported small negative impacts on traditional public school students’ tests that increased with the number of years traditional public schools experienced charter competition.30 Based on data through 2004–05, Scott Imberman reported small negative impacts on student achievement in one large southwestern district, but positive impacts on middle and high school students’ discipline.31
Again, more research is needed regarding the effects of charter schools on the achievement of students in nearby traditional public schools. Existing studies are limited to specific areas and may not be generalizable nationally. They have looked almost exclusively at standardized test scores — an important but limited outcome — and the most recent data they consider is from 2009. As the number of charter schools has increased nationwide, forthcoming studies can be expected to leverage more recent data and thus estimate impacts of increased charter school competition on traditional public schools.