The Narrow Focus of Pre-K Research
By Katharine B. Stevens
OP-ED
March 23, 2016
A couple of weeks ago, RAND released a report on public pre-K, presenting "a compilation and critical assessment of the most-rigorous evidence" on pre-K programs and reaching a familiar conclusion: High-quality pre-K works.
The report, like many that preceded it, is correct that dozens of pre-K studies show "favorable impacts." But here's the problem: The research showing these impacts is weaker than commonly understood, and the impacts shown are much less important than they appear.
For one, current research methods have crucial flaws. Most research on pre-K is conducted using two methods: regression discontinuity design and propensity score matching. Results found are only as good as the methods that find them. But few nonexperts know much about the fine print of these two methods, and there are important limitations to both.
The regression discontinuity method focuses specifically on a select group of children whose parents send them to pre-K. It assesses the impact of a year of pre-K on children's test scores in the first months of kindergarten. Children who drop out of pre-K are eliminated from these studies, so the outcomes reported in these studies are only for children whose parents sent them to pre-K and who successfully completed the full year.
Using this method, it's impossible to tell how much of a program's results are because parents self-select into the program and because dropout children are weeded out of the study. And since the majority of children don't attend pre-K in the first place, findings from these studies apply only to the limited group who do.
Studies using propensity score matching compare two groups of children: one group whose parents sent them to pre-K and one group whose parents did not. Except for this difference, researchers attempt to "match" the two groups on other variables, like race, income, family structure, neighborhood and so forth. If the pre-K group does better in, say, fourth grade, researchers attribute that to the pre-K program because the children are "alike," based on the variables they've identified.
But parents who send their children to pre-K are different from parents who don't, even if their race, income and zip code are the same. So the problem with matching studies is that it's impossible to tell whether better performance in fourth grade is due to children's participation in pre-K or to the kind of parents they have or to some combination of both.
Second, the impacts that studies measure are often of minimal importance. The RAND report, along with many others, concludes that "pre-K works." But the critical question is: works to do what? The positive results most studies report are significant from a statistical point of view but have little importance from a policy point of view. Very few studies examine what matters: children's long-term school and life success.
Instead, most pre-K research is narrowly focused on short-term gains in rudimentary academic skills – such as identifying letters of the alphabet, recognizing vocabulary words and counting small numbers – in the first months of kindergarten. Most so-called positive results show that kindergartners who attended pre-K are a couple of months ahead of their peers in these basic skills; in other words, children who went to pre-K know letters in September that they wouldn't have known until, say, November if they hadn't. But while these kinds of short-term gains in basic skills are easy to measure and look good in headlines, they clearly aren't what's important.
Overwhelming evidence shows that the key to children's long-term success is a range of cognitive and noncognitive capacities such as language and executive function skills, reasoning, critical thinking, problem-solving, persistence and the ability to get along well with others. Pre-K advocates claim that small gains in basic kindergarten skills lead to large gains in children's cognitive and noncognitive capacities which, in turn, lead to graduating from high school and staying out of prison. Maybe it works that way, and maybe it doesn't. But it's a huge assumption that simply hasn't been tested.
Pre-K is in the political spotlight and is a popular focus for research. But while we have scores of studies on the impact of conventional pre-K, usually for four-year-olds, the core policy question remains unanswered: What are the most effective early interventions for improving disadvantaged children's lives?
A stronger knowledge base is urgently needed to guide early childhood policy. To build it, early childhood research must focus on the most important questions instead of the most fashionable or convenient ones and pursue new approaches to rigorous, policy-relevant research.
See Also
Report ~ April 12, 2016
Op-Ed ~ April 26, 2016
Blog ~ February 25, 2016