Wednesday, August 9, 2017

Every Number Tells a Story

It's almost back-to-school time. So let's look at the Richardson ISD by the numbers. Specifically, the numbers from "K-12 school performance data at your fingertips." RISD's overall numbers can tell any story you want.

"Steady as she goes": RISD consistently ranks in the top half of all school districts in Texas.

"Sound the alarm": RISD dropped from the 86th percentile in 2010 to the 56th percentile in 2015.

"Back on Course": RISD climbed from 56th percentile in 2015 to 72nd percentile in 2017.

So have at it. What's your story?

When you dig into the scores for individual schools, the stories can get really imaginative. Like seeing animals in clouds.

Out of 4,308 public schools in's database, RISD's Brentfield Elementary ranks 26th. Thurgood Marshall Elementary ranks 3,966. Both schools in the same district, with the same district policies, same curriculum, same central administration. What could be different? Maybe you can spin a story about school administrators and teachers, but I'll probably look to demographic differences to explain these numbers. A classic story that's still good: Demography is destiny.

Year-to-year variability in rank is harder to settle on a story. One school climbed 1,318 places in one year. Another school dropped 1,232 places. In one year. Same school district. How do you explain that? Student turnover might explain some differences, but that much? New students coming in generally match the profiles of students leaving. They are often from the same household. Overall, in general, on average, I'd expect their academic performance to be similar. Just noisy data? Maybe, but those are some huge differences to explain by noisy data. Apples-to-oranges from year-to-year? Maybe, if the underlying tests changed from year-to-year, scores of different schools might be affected differently. If the explanation is major changes the district made, why would one school rocket upwards and another crash? I don't have enough imagination to tell a story with this data. Or at least a story that I am confident isn't just a fairy tale. Maybe the teachers, administrators and parents at each individual school can tell their own unique story. Or maybe the folks who have lost all faith in testing, ranking and rating in general have a compelling story.

Every number tells a story. Tell me yours.


Bryan holland said...

Always a fun debate on how risd is doing. From a statistical standpoint, one cannot compare data from two different areas without taking into account independent variable differences. The only story that can be derived is from the trends. I e data point is not a trend. Three data points begins one. From 2015 to 2017, all four high schools trended up. Risd in general is at 2013 level reversing 5 years. Lahe highlands moving from the bottom 18% in Texas to 53% is amazing. 2018 data will hopefully follow the same trend.

The "torch and pitchfork" party in risd used these same numbers a year ago to pronounce risd was burning and we needed a new superintendent. Perhaps like other things, another apology is in order.

Kudos to the district and specifically lake highlands. Definitely goat worthy.

Mark Steger said...

What makes year-to-year comparisons difficult is the constantly changing nature of the testing used. RISD Superintendent Jeannie Stone says,

"Straight comparisons from year-to-year cannot be made. Accommodations are removed, student groups are tested in different ways with different measures and applications, and all this muddies the waters when trying to help our kids. We cannot accurately assess the areas our students have improved and continue with those initiatives that are working, just like we can’t dig into student performance decline and make better those avenues to reach more kids."

bryan holland said...

agreed. when measuring year over year at the same school, that can be an issue, but when normalized against the average (which is what schooldigger does), it cancels out the variables . RELATIVE to the norm, i.e. Texas average, these comparisons are valuable and unfortunable the only real thing we have to go by.

IF we cannot accurately assess the areas our students have improved and continue with those initiatives that are working,. . . . then just what the heck are we doing?? Even if it's flawed, you have to measure, work against those measures, and account for the variable as a standard deviation, and live with it.

what gets done is what gets measured. If you don't measure it, it won't get done. Sorry STAAR haters. gotta have something.