Greater Greater Education

DC students may find themselves stumped by Common Core tests next year

Next year DC public school students will take a new standardized test that's supposed to test their critical thinking skills. But a number of questions on a publicly available practice test are confusing, unrealistically difficult, or just plain wrong.


Photo of test score sheet from Shutterstock.

A recent post on Greater Greater Education criticized the design of the Common Core-aligned test that DC will begin using next year, which is being designed by a consortium of states called PARCC. But judging from sample questions, there are problems with the test that are even more basic than its design.

Like the author of the recent post, I took the PARCC English Language Arts practice test for 10th-graders. I've been a writing tutor for a number of 10th- and 11th-graders in a high-poverty DCPS school, so I have some idea of students' literacy skills at that grade level.

The test consists of 23 questions, most of which have two parts that are based on a given text. I ran into problems with the very first question. The relevant passage, taken from a short story, read as follows:

I was going to tell you that I thought I heard some cranes early this morning, before the sun came up. I tried to find them, but I wasn't sure where their calls were coming from. They're so loud and resonant, so it's sometimes hard to tell.
Part A of the question asked for the meaning of the word "resonant" as used in this passage. The choices were:
  • A. intense
  • B. distant
  • C. familiar
  • D. annoying
I was stumped. None of these words matched the definition of resonant as I know it, which is more like "echoing." But, being a dutiful test-taker, I knew I had to choose A, B, C, or D.

Maybe this particular author was using the word in an idiosyncratic way? I tried to forget what I knew about the meaning of resonant and instead looked for what teachers call "context clues." From what I know, many DC 10th-graders would be in a similar position.

The passage said that the speaker wasn't sure where the cranes' calls were coming from, and the only word that seemed to square with that fact was "distant." So I chose B.

Wrong. The answer was A, "intense."

I suppose you could make an argument that "intense" makes sense here as a definition of resonant, but it's not obvious what it would be. Why would intensity make it hard to identify the source of a sound?

That was the most egregious example I found of a badly written test question. But I found a number of other instances where the correct answers were far from clear.

Arbitrary distinctions

The next set of questions, for example, concerned a descriptive passage about a firefly hunt. The passage described the fireflies as "sketching their uncertain lines of light down close to the surface of the water."

One question asked what was implied by the phrase "uncertain lines of light." I chose: "The lines made by the fireflies are difficult to trace."

Wrong. The correct answer? "The lines made by the fireflies are a trick played upon the eye."

Well, maybe. But there was nothing in the passage to indicate that one answer was any better than the other. Do we really want to make the results of high-stakes tests depend on such arbitrary distinctions?

Another literature-based set of questions was relatively clear, although not exactly easy. But one basic problem was that, right at the beginning, the author used the word "machine" to mean bicycle. If you put your cursor over the word "machine," which was underlined, and clicked, that rather unlikely definition popped up. But if you didn't happen to do that, it would be pretty hard to understand the passage.

Reading Supreme Court opinions

I did a lot better on a section where all the questions were based on excerpts from a majority and a dissenting opinion in a Supreme Court case about the First Amendment. But then again, I have a law degree, and, having spent a year as a law clerk to a Supreme Court Justice, I have a lot of experience interpreting Supreme Court opinions.

I suspect the average DC 10th-grader will have a much harder time with that section than I did. It's not that the questions call for extensive background knowledge in constitutional jurisprudence. In fact, they call for reading a text closely and making inferences based on that text, just as Common Core-aligned tests are supposed to.

But if a test-taker confronts a lot of unfamiliar concepts and vocabulary words, she's unlikely to understand the text well enough to make any inferences. In just the first few paragraphs of the majority opinion, she'll confront the words "nascent," "undifferentiated," and "apprehension." Based on my experience, few DC high school students are familiar with these words. Nor are they familiar with what the Supreme Court does or what the First Amendment is.

I certainly hope that someday DC 10th-graders will be able to understand texts like these well enough to use them to demonstrate their analytical abilities, and I hope that day comes as soon as possible. But only those who have no idea what's going on in DC's public high schools can believe it will come by next year. Most likely, students will either guess at the answers or just give up.

Many DC students, especially those in high school, are performing far below grade level. Even if their teachers manage to bring up their performance substantially, questions like these won't be able to measure that improvement because they'll still be pitched at too high a level.

Real test questions are secret

Of course, students won't actually be getting these practice questions when they take the tests next year. Nor did they get these questions this spring, when many of them participated in field tests. But because the real questions are closely guarded secrets, the practice questions are all we have to go on.

Recently the principal of a New York City elementary school complained that she wasn't allowed to reveal "the content of passages or the questions that were asked" on the Common Core-aligned tests given at her schoolalthough she did say the tests were "confusing, developmentally inappropriate, and not well aligned with Common Core standards."

Those weren't PARCC tests, but PARCC has an even more draconian rule about its field tests. According to its website, teachers aren't even allowed to see the test questions: "Items may not be viewed by anyone other than the students who are participating in the Field Test."

That may be why, in all the furor over the Common Core, we haven't heard complaints about mistakes or lack of clarity in the questions.

I'm not opposed to the Common Core, nor am I opposed to testing. But the success of any initiative depends on its implementation. And from what we can tell, so far PARCC's implementation leaves a lot to be desired.

No doubt it's hard to design a good multiple choice test, especially one that is trying to assess higher-order thinking skills. But at least some of the problems with the practice questions could easily be fixed. Given that PARCC has received at least $170 million to design these assessments, you would think they could have done a better job.

(The PARCC practice test I took also had 3 essay questions, which were pretty challenging. Scoring those will raise a whole other set of difficulties.)

Let's hope that the results of the PARCC field tests will bring these problems to light, and that PARCC will manage to fix them before students take the real tests a year from now. Perhaps it would help if other members of the public tried taking the practice tests for other grade levels to see if they suffer from similar defects. If you're able to do that, please let us know your impressions in the comments.

These tests are important. DCPS will use the results to evaluate teachers, and we'll all be relying on them to determine if efforts to improve education are working. Not to mention that students who take them may suffer a good deal of stress, even though the tests won't affect their grades.

We need to be sure that these are tests of what students are actually learning, not just guessing games.

Natalie Wexler is a board member at DC Scholars Public Charter School and a volunteer tutor in a DC Public School. She also serves on the board of The Writing Revolution, an organization that brings the teaching of analytical writing to underserved schools. She has been a lawyer, a historian, and a journalist, and is the author of three novels. 

Comments

Add a comment »

Zowsers! I tried some of the questions and they are not easy.

Also, becuase they are multiple questions based on one essays, if you misunderstand that essay you might really trip up 4-5 other questions.

I'm not sure of the time limits, but they seem to be questions designed to force you to be patient.

by charlie on May 5, 2014 12:01 pm • linkreport

Like everyone I would worry about national headlines like, "American Students Perform Poorly on New Exams," but won't the local relative differences remain relatively static due to students' varying ability to perform on grade level?

That is, if Question X is vague, won't most everyone get it wrong and the differences will wash out?

by ar on May 5, 2014 12:52 pm • linkreport

ar - Even if everyone in DC gets a certain question wrong, under PARCC the differences probably won't "wash out." That's because there are about 20 states plus DC participating in PARCC, and all of them will be held to a common standard. So the "cut score" -- the point at which "proficiency" is set -- won't be based just on the performance of DC students.

On the one hand, it's good that we'll finally have some way of comparing student performance across multiple states. On the other hand, DC -- which, unlike any state, is an entirely urban area, and therefore has a high proportion of relatively low-performing students -- is likely to come out looking pretty bad.

by Natalie Wexler on May 5, 2014 1:20 pm • linkreport

I have mixed feelings about what you are describing. DC CAS has been like kind insurance tables that don't really tell you you are overweight. I feel like the PAARC is going to be the tables that tell you, not only are you overweight, but we don't have many good tools to help you lose weight. All of these tests are by default knowledge tests. When you say most 10th graders don't know what the Surpreme Court is or does, I cringe. We have not for years focused on helping kids accumulate enough knowledge, instead we offer them too many random reading passages that pass as education starting in the very early grades. Yes the next several years are going to be rough for students, teachers, educational institutions, but if we want more capable students. We have to figure out how to get rid of the educational junkfood of test prep, find the main idea workbooks and get the equivelent of educational veggies, solid lit, real science and history.

I do hope they suspend connecting testing to teachers, but I do not want them to abandon these standards.

by DC Parent on May 6, 2014 11:36 am • linkreport

What baffles me is why school systems (DC is not alone in this) are not trying to figure out (1) at what grade level students are actually achieving, and (2) how individual students are improving over time.

I think there has been a GGE article about this, but why on earth is our test not being written to assess actual grade level of each student? Computer algorithms can and have been written that respond to a tester's answer and adjust future questions accordingly (so for educational testing, the questions would start slightly below grade level and the program itself would raise or lower the difficulty of subsequent questions based on the student's answers). That way, kids are tested on what they actually know, and we have a meaningful way to track progress over time.

Right now, a child in 3rd grade who tests at "basic" has twice the ground to make up in 4th grade to get to proficiency. Essentially, because they did not learn significant portions of the 3rd grade curriculum, they are attempting to learn 4th grade material without the 3rd grade information it's based on. So, if that child again scores "Basic," the score could be interpreted to means one of two things. Either it shows that the child learned enough of the missing 3rd grade material to successfully tackle at least some of the 4th grade material or, it shows a child who has not made any academic growth, because the child was on "Basic" two years in a row.

With progressive testing, you might be able to see that although the child was on a 2.4 grade level when taking the 3rd grade tests, s/he had moved up to a 3.7 level when taking the 4th grade tests. More than a year's growth over the course of a school year would be something to celebrate. However, there is nothing about the current or proposed testing regime that would allow this level of assessment.

As a DCPS parent of a child in a Title I school, this testing means that we're not collecting the data we need to be able to reach those of our children who are struggling. It is the embodiment of testing simply to test.

A disproportionate percentage of the kids coming from Murch and Lafayette and Janney will be fine, no matter what the tests look like. But if we're not using tests to let us see, in a meaningful way, what is working for a child and what is not, the achievement gap between kids from vastly different socioeconomic statuses will continue to grow.

by LMO on May 6, 2014 2:54 pm • linkreport

How many DC children have ever seen a crane or fireflies over a pond? For that matter, you won't find cranes and fireflies in arid climates either, so these questions will have no point of reference for kids in much of the country. Test questions must be based on universal common experiences to be valid, and these example are not relevant for much of the test-taking audience. It reminds me of the old question: There are three crows on a fence. The farmer shoots one. How many are left? The correct answer is none. Every country kid know that birds fly away at the sound of gunfire.

by Susie on May 7, 2014 9:39 am • linkreport

It's hard to imagine who is really developing these tests in the 20 states plus D.C. I imagine there are some puffed up egos in the group. Evidence? The need to capitalize "field test." Good luck convincing such folks to make any changes, regardless of how right you are.

by Econista on May 7, 2014 12:33 pm • linkreport

Look at the cranes question. It is not so much finding an exact-match definition as it is a matter of inferring from context which of four possible synonyms is the best. When you think of it that way, it is not hard for a kid who has been taught reading comprehension to figure out which of the four choices fits best. "Intense" was the best choice because the test prompt emphasized the noise level. "Distant" (B) doesn't work for just that reason. I agree that the not-so-smart students might pick B, but that's OK - they can get a job writing opinion columns on the Internet!

by LMA on May 7, 2014 7:58 pm • linkreport

LMA -- At the risk of repeating myself, let me explain why -- judging from the context -- "intense" is not a good match for "resonant" on the test. You say "the test prompt emphasized the noise level." That may be correct if you just look at the word next to resonant, which is "loud."

But "inferring from context" shouldn't mean just looking at the closest word in a given text. Rather, readers should look at the passage as a whole to glean clues as to an individual word's meaning.

In this instance (as I mentioned in the post), I would suggest looking at the REASON the speaker mentions that the cranes' calls were loud and resonant. If you do that, you find that he was explaining why he couldn't tell where the calls were coming from. There's no reason why loudness combined with intensity would make the calls hard to locate -- at least, not one that I can see. It's only if you define resonant as "echoing" that the word makes sense, in context.

Obviously there are some people who would find "distant" an obvious choice here. They can always get jobs writing test questions -- and apparently they have. But is that what we really want?

by Natalie Wexler on May 8, 2014 1:43 pm • linkreport

@Susie-

The problem is that we need children that can imagine what fireflies looklike over a pond. If we neutralize everything to universal experience, it becomes banal. I have never been to a rain forest, but it does not stop me from reading about or imagining it. Yes we need to build children's experiential vocabulary, but we should not sanitize what they read or are tested on because they might not have experienced it.

by DC Parent on May 9, 2014 10:40 am • linkreport

I have written quite a bit about the Math Wars on my blog this past year: http://jennyhatch.blogspot.com/p/homeschooling-for-math-excellence.html

by Jenny Hatch on May 10, 2014 10:29 am • linkreport

In the example asking for the meaning of "resonant", I agree with the author that the question is an extremely poor use of the word itself, and the question would have left me feeling extremely uncertain of the desired answer.

However, high intensity of a sound can often make it extremely difficult to track to source, as anyone who has ever tried to determine which of five smoke alarms in a house needs the new battery.

by Yancey Ward on May 10, 2014 10:51 am • linkreport

Every student should know by 10th grade what the Supreme Court is and know generally that the First Amendment guarantees the right to free speech. I grew up in Maryland where there is a 9th grade civics requirement, so even if I haven't learned about it through studying American history in elementary and/or middle school, then I would have learned about it in civics - does DC not have a civics requirement?

I took the 10th grade test and I only had issues with 2 of the questions (the firefly one and one other). I also remember standardized tests being somewhat confusing when I took them growing up in the 80s and 90s - is this test any worse than what kids have been taking? (I've never taken the DC CAS, are sample questions available for the reading portion? I only see math and science on the OSSE website).

As for the "resonant" question, I can see how this could be confusing because the author of the passage was trying to say that the sound of the cranes was so loud (as in Yancey's example of the multiple fire alarms) that it was hard to pinpoint, yet it also said that the father couldn't see the cranes, and this could be interpreted to mean that they were far away. Either way, I associate the work "resonant" with "reverberating" and “booming” so if a student knew what the word meant to begin with, they could have gotten this answer correct.

by grumpy on May 12, 2014 9:13 am • linkreport

[This comment has been deleted for violating the comment policy.]

by Elijah on May 14, 2014 7:07 am • linkreport

You were too kind. Those literary questions are also badly written. "I was going to tell you that I thought I heard..." is dreadful.

Seeing this nonsense, I'm reminded of the IQ tests that were given to draftees in World War I. Those were biased in favor of those who grew up rich and in large cities, much as these are biased in favor of the teaching schemes behind Common Core. Even worse, it's now believed that, because many couldn't make sense of those questions, they simply gave up and didn't try.

One result was a post-WWI hysteria about the 'menace of the feeble-minded' that led to nasty immigration restrictions and forced sterilization. I can't help but wonder if there's an agenda behind these tests. Do they actually want bad results or do they want bad results so they can get something else?

Wiser people gave those WWI tests to successful businessmen and community leaders and discovered that they scored equally bad. The problem wasn't with our nation's intelligence. It lay with those who'd written the tests, much like parents taking Common Core tests.

Were I the publisher of educational materials, I wouldn't be creating Common Core materials. I couldn't compete with all the influence giants like Pearson can buy. Instead, I'd be creating and promoting unique products as not being Common Core compliant.

One suggestion for parents. If you're worried about schools teaching badly, use the Scout Technique. In To Kill a Mockingbird, Scout can't be taught to read the dreadful 'whole word' approach because her father has already inadvertently taught her phonics. Do the same. Teach your kids to add, subtract and the like before they have to face Common Core's weird ways.

--Michael W. Perry, co-author of Lily's Ride: Rescuing her Father from the Ku Klux Klan

by Michael W. Perry on May 20, 2014 4:24 pm • linkreport

In the following passage, determine the closest meaning of the word return.

Any person who willfully aids or assists in, or procures, counsels, or advises in, the preparation or presentation, in connection with any matter arising under this part, of a return, affidavit, claim, or other document that is fraudulent or false as to any material matter, whether or not the falsity or fraud is with the knowledge or consent of the person authorized or required to present the return, affidavit, claim, or document, is guilty of a misdemeanor punishable as provided...

This is the core problem with the standardized tests. Everyone commenting here knows what the word return means in the contexts that they know it to be used. If the word is used in a context of which they have no knowledge, then they guess at its meaning.

Many students do not live in nor experience the context of the passages and questions of the standardized tests. I understand it may be difficult for some to fathom this, but that is the reality. Children who do not know what fireflies are May think the passage is referring to a laser light show, in which case none of the possible answers are understandable. They may think a crane is a machine used on construction projects, in which case the passage itself is nonsensical (construction sites do not appear overnight and cranes do not run pre-dawn)

In my first example the passage is part of tax code. Now what does return mean to you? Same passage same question, but isn't it all so much clearer now? The return is not just any document, it's a tax return document. It has a richer, fuller meaning because yearly we suffer through completing this document.

To understand the problem with this testing, one needs to understand that American students do not have the same world view, environment, culture, language, experiences and education. Because of those limitations, much of the test is simply nonsense language to a large number of them. For many students, scaffolding would be required for them to demonstrate what and how we'll they know the standards. What do you think the word scaffolding means in this context?

by Concerned Parent on May 30, 2014 4:37 pm • linkreport

Add a Comment

Name: (will be displayed on the comments page)

Email: (must be your real address, but will be kept private)

URL: (optional, will be displayed)

Your comment:

By submitting a comment, you agree to abide by our comment policy.
Notify me of followup comments via email. (You can also subscribe without commenting.)
Save my name and email address on this computer so I don't have to enter it next time, and so I don't have to answer the anti-spam map challenge question in the future.

or