Here We Go Again

Here we go again. In Tennessee, like many states in the Union, we test our students using standardized tests every April. In May and June, the results come out and the questioning begins. The last couple of years have seen the questioning get louder and louder. Last year, it was over a delay in test results. This year, teachers are taking to social media, questioning the quick scores because they seem abnormally high. Are they, though?

Well, if you are like most parents, you probably don’t know what a quick score is. I certainly didn’t. So I went to the Tennessee Department of Education website for a definition. I didn’t find a definition, but I did find this:

Q47: Should I use my district’s/school’s quick scores for accountability determinations?

Due to post equating and psychometric reviews on assessment data, quick scores might look different from final accountability results. Districts may aggregate their numbers for their own data analysis; however, these are merely estimates. There is always the potential for changes in scoring. In all cases, we do not keep a record of students for whom scores change. Quick scores are embargoed which means they are not meant for public dissemination.

So that sounds pretty serious, but no more clear than it was before I checked. One thing was clear – that in order to interpret the quick scores you needed to know the cut scores. You got it, back to the web site.  That’s where I found this: The index cut scores are an estimate of the number of items the student would be expected to answer correctly to achieve basic, proficient, and advanced designation if there had been 100 such items for each category.

Okay, that’s not much clearer, but it’s clear that I needed the cut scores in order to assess the value of the quick scores. Clearly, this isn’t like Ms. Johnson’s 5th grade class where there were 50 questions and each one you got right earned you two points, and if you got between 80 and 90 points you got a “B,” 90 to 100 an “A,” and so on. Under the state scenario a student receives a score, say an 87, and what ever number the cut score is set at determines whether the student is proficient or advanced. If the cut score is set at 88, the child would be considered basic. If the cut score is 86, they are proficient. Sounds fairly simple, but there are questions about how cut scores are derived at.

But let’s put aside any suspicions and ask, where are the cut scores? Well, they aren’t available yet because arriving at them means some additional calculations need to occur. Well of course they do, didn’t we have a similar issue last year? Instead of releasing the cut scores, on Friday, Education Commissioner McQueen released this cryptic letter explaining why they weren’t available:

Directors,

I want to thank you for your work in finalizing student demographic and teacher claiming information to close this year’s TCAP cycle. I know many of you have received your quick scores for student grading and are anxious to understand more about your district’s overall performance. Though the department made the decision in 2014 to stop associating TCAP performance levels with quick score results, we do want to provide information as accurately, transparently, and quickly as possible.

To that end, the division of data and research will provide a detailed communication regarding quick score use and interpretation in our May 27 Director Update, followed by a release of preliminary data regarding quick score relationships to raw scores and cut scores to determine proficient (versus non-proficient) on June 1. For now, I caution you to avoid communicating any results regarding proficiency rates based on the 2015 quick scores using performance level relationships that were last calculated and communicated in 2013.

Quick scores are generated for use in student grading only. As such, there will not necessarily be a consistent relationship between quick scores and performance levels for achievement from year to year. Performance levels are determined first by raw score to scale score conversions and then through cut-scores defined by the standards setting process. Over the next couple of months, we will engage our TOSS working group for accountability in further conversation about how we address quick scores during the transition to TNReady. In the meantime, please look for the memo in the May 27 Director Update and the follow-up information on June 1. As a reminder, we will also include this timeline in today’s Director Update.

Best,

Candice

(I like the way she signs her memos “Candice.” After all, she’s just one of us, right?)

I’m not going to try and decipher exactly what all that means. It is obviously way more complicated than I can handle, so I’ll leave that to smarter minds than mine. However, I do have a couple thoughts on our student testing system that I’d like to share. Some things that I do understand and I don’t believe can be said enough.

Critics often say that we should run our school system like a business. Well, you can pick up any number of business books, and they will stress the value of having an evaluation system that stake holders all buy-in to. Without that buy-in, there is no value. If people don’t believe in the fidelity of the system, it becomes too easy to attribute outside factors to the results. In other words, they start to feel that data is being manipulated to augment an agenda that they are not privy to and not included in. I’m not saying results are being manipulated or not being manipulated when it comes to our student evaluation system, but I am saying that there seems be a growing belief that they are, and without some kind of change, that perception will only grow. I’ve always maintained that perception is nine-tenths of reality.

Candice is the one person who has the ability to reclaim that belief in the system. Imagine if she were to announce, that based on her listening tour, she is calling for a complete review of our testing process and its timelines. She recognizes that in its current configuration, the testing process is not meeting the needs of parents, administrators, teachers, or students. Timelines could be adjusted to ensure that when scores are delivered, all components are delivered and no one will have to speculate what they mean. Definitions and processes could be clarified so that all could understand the results without relying on “experts.” Because as it stands, nobody can give you a clear picture of what it all means, yet we spend an inordinate amount of time focusing on the test. (The below chart was created by a parent.)

That’s a lot of focus on something that nobody can give a clear concise explanation of. What if parents were given copies of the test their children just took and were permitted to review these tests with them? What if teachers were permitted to see tests and judge the strength of the test questions? What if teachers were given Unless the process was truly flawed, greater transparency could only increase confidence in the way we test our children. Perhaps if we spent as much time defending stakeholder’s proprietary rights as we did testing companies, some faith would be restored.

Which leads me to my second thought and one that cannot be understated. We need to make sure that the public has a full understanding of exactly what a bell curve is and that it serves as a base for our measurement system. In a new posting, Jersey Jazzman, an education researcher and blogger, does an exceptional job of explaining how it all works. A bell curve is applied to the test results, and that means that some students will fall to the exceptional end of the scale and some students will fall to the other side, but the majority will fall in the middle. Let me spell that out further. While we are demanding that students do exceptional work, we are utilizing a measurement system that guarantees the majority of them will do average work. Because if too many of them do exceptional work, we’ll just make the test harder so that the majority fall back in the middle.

Let’s take it a step further. Since the evidence is pretty strong that standardized tests are a better measurement of socio-economic status than actual learning, who do you think will fall to the lower end of the curve? That’s right, our low income children and our children of color. Reformers will latch on to these scores to create strategies like charter schools, extended school days, and no-excuse discipline policies.  They will apply these strategies predominately to our low income children and children of color.

I recently brought a magician to a high-poverty elementary school to entertain the children. A teacher came up to thank me and made the comment that they don’t often get opportunities like this one due to the testing scrutiny they are subject to. Kids in poverty get more focus on what is measurable while wealthier kids enjoy a more well-rounded education. When was the last time you heard a call for longer school days in a wealthy district? What about the need for a no-excuse discipline policy? A test that just reinforces socio-economic status will just serve to create two separate and unequal education systems.

At a listening session for the upcoming Project RESET in Nashville a roomful of education professionals offered very insightful assessments of our educational system when the owner of a local charter school spoke up and proclaimed that we can talk about engagement, parental involvement, and diversity all we want, but we need to focus on achievement because 13 percent of students are not proficient in reading and to her that was unacceptable. To me, it sounds like she has a lack of understanding of how the system is designed, and that based on that design, the system is working. Because if that 13% ever became proficient, we would just adjust the test to make sure that either they or some other students fell back into that non-proficient status. Starting to feel like it’s a rigged system? It should.

At a luncheon the week before, in front of hundreds of civic leaders, Nashville Public Education Foundation, a main sponsor of Project RESET, president Shannon Hunt proclaimed that all students have the identical potential to achieve. Apparently, she hasn’t looked too closely at our measurement system because under that system, only a few have the chance to achieve at a high level, lest we, again, adjust the test. Many in the room will use those words to justify further privatizing a system that is already under attack, and again it will be our poor children and children of color who will be affected.

In Tennessee, we have the Achievement School District, which has the mission of taking the bottom 5% of schools up to the top 25% while ignoring the fact that there will always be a bottom 5%. The use of a bell curve goes one step further and insures there will always be below average schools and teachers. Or, as the reform movement likes to label them, failing schools and failing teachers. How many parents and community members are aware of this, and how many take the results at face value? Since NPEF is sponsoring Project RESET to reset the educational conversation in Nashville, this discussion of test scores and failing schools might be the area to start the reset. Perhaps we could have an honest conversation about what standardized test tell us.

Students are not alone in being held to a higher standard in a system that only allows a high output for a few students. Countless articles have been written about the need to have a great teacher in front of every class, yet each of those teachers will only be allowed to produce average outcomes. Because again, if all those teachers produced great outcomes, we’d have to redo the test to make sure those outcomes were average. I can hear the critics shaking their heads now. You’re oversimplifying it, they’ll say. They’ll point out that’s why there is an observational portion and a growth element added to the teacher evaluation system. And what happens when that observational portion produces results different than the test? We say the observations are biased and we re-address the scores.

It’ll be interesting to see how it all plays out this summer. I don’t know if Tennessee needs high test scores to fortify the narrative that reform is working or low test scores to reinforce the need for the move to Common Core. I suspect I’ll know by the end of the summer. I do know that at some point the state will have to address the growing lack of faith in our evaluation system. At some point we will have to stop demanding that children produce at a high level in a system that guarantees the majority will produce at an average level. We will have to face facts that our evaluation system aids in the creation of inequalities and fails to give us a true picture of student outcomes. That is perhaps Candice’s biggest challenge.

---

‹ UNION STRONG

A Pleasant Surprise from a Politician ›

---

Categories: Uncategorized

Tags: #EdNetBlog, #ExpectMoreTN, #parents, #Teachers, Education