Test scores rise. Is it better education or gentrification?

DC students' scores grew dramatically on national standardized tests, more than almost every state. This is good news, but it's hard to know whether the gains actually mean we're educating students better, or just that more wealthy families are sending their kids to DC schools.

Photo by gak on Flickr.

The Washington Post editorial board isn't hesitating to claim credit on behalf of its agenda. "School reform in the District is working. That is the unassailable message" of the news, the board writes. "The NAEP is the gold standard or, to use [US Secretary of Education Arne] Duncan's description, 'irrefutable.'"

I hope school reform does work, and in particular, that it actually helps struggling students from disadvantaged backgrounds get a better education. But good as the NAEP may be, there's nothing "unassailable" or "irrefutable" about the Post's conclusion thus far.

This is "misnaepery"

That's because the NAEP, the National Assessment of Education Progress, just measures 4th, 8th, and 12th graders every two years. The problem is that one year's 4th graders might be very different from the last. Since wealthier students tend to do better on standardized tests, one reason for the rise in DC's test scores could be that more affluent kids are taking the NAEP.

Steven Glazerman coined the term "misnaepery" to describe using NAEP data to draw conclusions beyond what the scores really show.

It's not that DC's test scores haven't gone up—they have. And it's always better to have higher scores than lower ones. But it's misusing statistics to say that this proves there's been a change in school quality, or that any particular leader deserves the credit.

What do we know?

We don't have data showing that the public school population in DC, as opposed to the general population, has become wealthier since 2011. But, as Emma Brown reported in the Post, "The demographics of test-takers in the District has shifted during the past two decades, with the proportion of white and Hispanic students growing as the proportion of black students has fallen."

It's possible that some of those Hispanic students are learners of English as a second language, the one group that failed to show gains on the NAEP. But presumably many of those white students are relatively affluent.

And while it's good news that the scores for all racial groups rose, the gulf between those groups remains wide. As Brown wrote, "Black and Hispanic students made gains, and achievement gaps between them and white students narrowed slightly in some subjects and grade levels. But the gaps widened or remained the same in others."

There's no free lunch for these statistics

It's also become more difficult to focus on the trends among students who qualify for free and reduced-price lunch (FRL), the most common method of identifying poorer kids of any ethnicity. The reason, as Brown explained, is that DC has changed its FRL rules.

Instead of requiring families to turn in forms showing household income, DC now provides free lunch to all students at any school where at least 40% of students are in foster care, homeless, or receiving welfare benefits. That could mean that some wealthier students at those schools are now lumped into the FRL category.

Brown quotes the head of the agency that administers NAEP as saying that the rule change is "masking whatever is actually happening." The official, Brown says, "cautioned against drawing conclusions about the progress of poor children in the District based on the 2013 test results."

Officials are touting that students who performed in the bottom 10% of test-takers have shown the most significant gains since the last round of NAEP tests. Because test scores are largely correlated with income, those are likely to be the kids at the bottom of the socioeconomic scale, which could be a ray of hope. But it's still not possible to tell from this data whether the actual poor students are doing better, or if this is more "cohort replacement" where this year's bottom 10% is different, and wealthier, than last year's.

We still have the wrong data

Maybe DC schools have improved, and we all certainly hope so. But to be sure that's the case, we need "longitudinal" data, which compares the same students from year to year instead of starting over with a new cohort. And unfortunately, that's not what the NAEP scores give us.

With all the testing that's happening, testing which often distracts from actual instruction, you'd think that we should be able to actually draw meaningful policy conclusions. It's a shame that there's so much data but not the data we need.

We do know that the number of higher-income families and students living in DC has increased. That can be good if we have the right policies to ensure that the rising tide lifts all boats instead of just a few. But we can't simply skip that necessary step.

Scores can go up if schools get better. Scores can also go up if wealthy families push out poor ones from the city or displace poor families from high-performing schools. We need to make sure any new education policies are responsible for rising test scores rather than the change in our population before celebrating too loudly.

David Alpert is the founder of Greater Greater Washington and its board president. He worked as a Product Manager for Google for six years and has lived in the Boston, San Francisco, and New York metro areas in addition to Washington, DC. He now lives with his wife and two children in Dupont Circle. 


Add a comment »

Let's not pat ourselves too much on the back. It's really embarrassing that DC spends more per child than any other state in the United States and we are always at the bottom of the heap in terms of schooling the young ones. Not a pretty track record at all. Although change takes place, sometimes slowly - and a change for DC education has been in the works for a few years, it could be that we are seeing some of that here. It is also probably that more and more people that are returning to live in DC and therefore school in DC, some who are wealthy. Wealth, however, does not signal the cause for such improvements in the school system (I am sure this point will be argued ferociously). I don't think it's an either/or paradigm but probably a combination of a number of factors. Although you could argue and point to the deficits of talented teachers, administrators in the DC school system (and I'm sure there is some truth to this), the number one factor, I believe, that influences the success of a school system is parental involvement. Sure, there are children who excel even without parental guidance/support/involvement but I think that is much more rare than the common occurrence. Whoever is putting their children into the DC school system, I am hoping and guessing that there is a trend to greater parental involvement in the schooling, which will also continue this upward on onward trend of an improving DC school system.

by DCAnon on Nov 8, 2013 11:36 am • linkreport

It's not wealthy parents that are improving the school system, but parents that actually care and convey the value of education to their children.

by Carlo on Nov 8, 2013 12:04 pm • linkreport

"It's really embarrassing that DC spends more per child than any other state in the United States..."

No, actually it's not. First of all DC is required to provide a combination of state *and* city services. Second, DC's extremely high special education spending is a function of extreme concentrated urban poverty. No other "state" is saddled with a similar demographic.

by oboe on Nov 8, 2013 12:54 pm • linkreport

Educational attainment of the parent is a better data point that affluence. There are plenty of studies that show children of parents with high education do better in school and on standardized test.

by Veronica O. Davis (Ms V) on Nov 8, 2013 1:55 pm • linkreport

This isn't definitive on whether the DC school population is growing more affluent, but the Kids Count data center just released figures on the number of children living in poverty in DC. For 2012, the figure was 29,000, or 27%. The year before it was 30%, so it does seem like the school-age population might be getting wealthier. On the other hand, if you go back to 2008 it was 26%. It was 29% in 2009 and 30% in 2010. So I'm not sure exactly where that leaves us. Here's a link to the data: http://datacenter.kidscount.org/data/tables/43-children-in-poverty?loc=10&loct=3#detailed/3/any/false/868,867,133,38,35/any/321,322

I agree that wealth is an imperfect proxy for parental involvement and degree of education, but it's the only one we have at the moment.

by Natalie Wexler on Nov 8, 2013 2:28 pm • linkreport

While the Post editorial may go a bit overboard in terms of praise, I think this post may be a bit too skeptical.

There are a few reasons behind my hesitation to buy into the "improvement=gentrification" theory.

1. While the demographics of students going to DCPS and charters are changing, those changes are primarily targeted in the elementary grades right now, and I would guess without looking at the data, that it's mostly at the P-3 levels. So while you may see some impact at the 4th grade level, there is probably a lot less of a demographic shift at the 8th grade level at this point.
2. If all quintiles are showing improvement, then it's not as likely as one might think that improvements in the bottom 10% are gentrification-driven. IIRC, many or most of the families in gentrified areas that move out to PG County and other suburbs are not at the bottom of the income scale (and may not be at the bottom of the educational results pile either).

The NAEP clearly has limitations in terms of how much it tells us, but so does cohort data (e.g., you're measuring the same students, but not measuring the same knowledge, from year to year).

by Jacques on Nov 8, 2013 4:10 pm • linkreport

If there is no parental support, there is a limit to what the school can do.

Most White and Asian kids have two married parents. This is not the case for black students and is the case for only half the Hispanic kids.

by Toady on Nov 9, 2013 12:21 pm • linkreport

A few assumptions worth challenging:
-'Smaller proportions of DC students living in poverty equates to low-income students being replaced with higher-income students in the school system (i.e., 'gentrification')' versus an increase in the household income of the same previously low-income students. I would expect some of the later is at work as DC has emerged from the recession (adult and child poverty rates have declined over the last 3 years).
-'No longitudinal data on students exists.' Yet a large component of the IMPACT assessment of many middle school teachers does measure the improvement of students over time (I am not aware how publicly accessible this information is however).
-DC's spending on education is comparable with 'other' states. DCPS spends hundreds of millions of dollars annually on special education funding, with large amounts devoted to enrolling students in private institutions at public expense (I don't believe other state school systems operate under such a burden).

I agree it is not reasonable to attribute improvements in DC's public schools to any particular leader - NAEP scores have been increasing since before the mayoral takeover and enrollment has risen since before Henderson became chancellor. But I think all of the signs do suggest that school reforms (as varied as they are) have been working.

by Brett on Nov 9, 2013 2:24 pm • linkreport

Does it matter why? Let's celebrate that it's happening (finally)

by Dad on Nov 9, 2013 6:21 pm • linkreport

Why do test score stories always discuss the so-called racial gap in test scores, but never discuss the gender gap? Is it because showing that girls consistently outscore boys turns the patriarchy narrative on its head?

Also, why doesn't anyone discuss the usual gap between Asian students and white students? Again, because it doesn't fit into a convenient Eurocentric narrative?

And why do we focus on grouping test scorers by complexion anyway? What is the possible practical usage for that information?

by spirit equality on Nov 11, 2013 11:17 am • linkreport

To Dad, while I understand your sentiment, it does matter because if we want to replicate or continue improving the school system, it helps to understand why, so we can do more of what works and less of what does not work.

To spirit equality, when people discuss complex issues, people like to simplify things, which is not a bad thing, but sometimes it then oversimplifies things. When education is broken down by what seems to be complexion-based discussions, it is not saying that Asian or Caucasian people are inherently smarter or more intelligent. Race really is a discussion about the intersection of society (and what is wrong with society, at times) and class and family structure are so intertwined with discussions of race and gender, in my opinion.

The caution is that there are always exceptions and generalizations sometimes, well, generalize too much in non-helpful ways.

by DC Anon on Nov 11, 2013 12:50 pm • linkreport

"Race really is a discussion about the intersection of society (and what is wrong with society, at times) and class and family structure are so intertwined with discussions of race and gender, in my opinion."

1) Gender is never included in these discussions (probably because to do so would require noting that girls consistently outscore boys, raising other problematic questions about our society, such as how girls can outscore boys then grow up to only earn a fraction of what they do in a so-called meritocracy)

2) Class and family structure don't have to be intertwined with discussions of race and gender, especially in the context of taking tests. Asian students outscore wealthier white students. Poorer students who were read to as children outscore wealthier children who were essentially raised by the TV. Grouping students by race is nonsensical and unhelpful. There's no rational justification for it, just a reflection of our society's insane obsession with race. If we could group stamp collectors by race, we probably would.

3) I also find it comical that some races don't even get tracked. How are Arab students scoring? No one cares, I suppose. Ha.

by spirit equality on Nov 11, 2013 5:07 pm • linkreport

Actually, gender does show up in discussions about education and test taking. Girls tend to outperform boys on tests until the upper levels. There could be a number of reasons for this.

Class and family structure don't have to be intertwined with discussions of race and gender; however, to ignore any relationships would be to ignore reality. A number of people seem to think (and I am one of them) that parental involvement is one of the best indictors of success in terms of schooling and education. If there is a single parent/broken home or parents are not around because they are working or working multiple jobs, then race and/or gender need to be a part of the discussion because they are factors that play a role in our society. To ignore some of these factors would not to be looking at the whole picture.

by DC Anon on Nov 11, 2013 5:37 pm • linkreport

Add a Comment

Name: (will be displayed on the comments page)

Email: (must be your real address, but will be kept private)

URL: (optional, will be displayed)

You can use some HTML, like <blockquote>quoting another comment</blockquote>, <i>italics</i>, and <a href="http://url_here">hyperlinks</a>. More here.

Your comment:

By submitting a comment, you agree to abide by our comment policy.
Notify me of followup comments via email. (You can also subscribe without commenting.)
Save my name and email address on this computer so I don't have to enter it next time, and so I don't have to answer the anti-spam map challenge question in the future.