Research designs for studies looking into charter school student outcomes look a little different than studies that look squarely at district school students or all public school students. That’s because charter schools are different.
Charter schools are schools of choice and that means that every student who enters a charter schools’ doors has self-selected to be there. The key to identifying the effect of that decision is finding a student who wanted to be there but is not. How researchers identify that student is an important and often contested process. But, while there is certainly a healthy debate around preferred research designs, samples, and outcomes, the positive effects of charter schools on their students are robust.
There are two widely used methods: “gold-standard” randomized-assignment designs and quasi-experimental designs. The key difference between these two approaches is how they account for the influence of selection bias on estimated results.
Randomized-assignment studies leverage lotteries to match those students selected into a charter program to those students that were not. Unobserved factors that influenced an application, like parental engagement are controlled for by the lottery and researchers can also ensure they are comparing similarly situated students across those groups. The difficulty is what these studies can say about the charter sector more generally since there may be something different about charter schools that have a waitlist or the area in which they are located. However, a recent meta-analysis gives us confidence that the results found in these studies can speak more broadly about the charter sector generally.
Quasi-experimental studies do not leverage lotteries but instead try and match a student attending a charter school to an otherwise similar student who did not. While this allows researchers to evaluate effects across a wider range of charter schools, they must assume that matching on observable characteristics can account for the possible effect of the unobserved family and student factors that may have influenced the decision to attend a charter school. While this assumption is somewhat tenuous, recent research (here and here) has emerged that give these studies further credibility. Furthermore, there are other considerations such as cost, coverage, and timing that make these studies useful.
At the end of the day, some studies are better executed than others. And although studies differ in design, sample, and outcome, we continue to hold charter schools accountable for student performance and we continue to see positive results for students in charter schools. This week, for National Charter Schools Week, I’m going to dive into the latest research on charter schools utilizing both of these research methods.
Nathan Barrett, Ph.D., is the Senior Director, Research and Evaluation at the National Alliance for Public Charter Schools.
