For the last two days I’ve been griping about standardized tests, brought on by this article and Diane Ravitch’s reaction to it.  I hope I’ve adequately demonstrated that relying on a pass-no-pass model for determining effectiveness in schools is at least pointless and at most actively destructive.  I’ve also talked about the alternative to a pass model, which is a growth model, and offered some criticisms of how growth models, at least as they’re practiced in Indiana, tend to work.

Here, I’ll present an outline of how a growth model for standardized testing ought to work.

• First, and most importantly:  remove any notion of “passing” and “failing” completely from the testing process.  The two most well-known standardized tests in America right now are the SAT and the ACT, the two tests for college readiness, taken by nearly every high school student at some point or another.  Even kids who don’t necessarily plan on going to college take at least one of those two tests, and many take both.  Have you ever heard of someone “failing” the SAT?  No.  Because it can’t be done.  You can get a terrible score on it, yes, but you can’t fail.  Your score is your score.  As it stands right now, creating a cutscore for “pass” and “fail” does the following:  1) It makes the test scores easier to manipulate (just change the cutscore and it looks like more kids passed– or that you’ve demonstrated “higher expectations”) 2) it puts an artificial, pointless barrier between kids who barely passed and kids who barely failed (there is no difference between a kid who got a 450 or a 46o; that’s a question or two.  It’s I-had-breakfast-and-eight-hours-of-sleep versus I-sorta-have-a-cold-today.  But if you put the pass cutscore at 455, it looks like a huge difference.  3) It embeds a shaming mechanism into the test that has no good reason for being there; 4) It creates an incentive for teachers to focus solely on the “bubble kids;” 5) It provides no useful information to anyone that the actual scores did not already provide.  There’s no reason for these tests to have a passing score. It is an entirely useless piece of information.  I can think of only one exception, which is when districts use test scores as part (PART!!!) of a decision on whether to pass a student from one grade to another.  Most districts don’t do that, though, since frequently scores aren’t available until very late in the year– it’s the second week of July already and I don’t know my kids’ scores yet.
• Removing the notion of pass/fail from the equation makes it easier to focus on growth as the metric.  As I’ve demonstrated already, this means that you can’t exclude any of your kids as “unimportant” to your school’s or your classroom’s end-of-year scores.  How a student’s score changes from year to year becomes vastly more important than what their score actually is, which is as it should be.  There’s a bunch of ways to do this; Indiana’s model has some good points but is unnecessarily complicated.  Here’s my suggestion:
• Pick a start year; any start year.   Divide those kids into groups based on percentile scores on the test.  I like using decile groups (in other words, ten) but you can use quintiles or quartiles or whatever.  In Year Two, determine how much those kids moved in their test scores from year one to year two.  There are a bunch of ways to quantify this depending on how mathy and technical you want to be about it; the simplest way is to determine movement by thirds.  In other words, let’s say the lower third of decile A went from a drop of 140 points to a gain of 10 points, the middle third went from a gain of 11 points to a gain of 90 points, and the top third went from a gain of 91 points to a gain of a million points.  You could use standardized deviations from the average or something else if you wanted, but the point is there’s a different standard based on your decile.  This means that the kids in the top decile (who don’t have a lot of movement up left for them) can only gain a few points or possibly even lose one or two and still be “high growth,” and kids who start in the low decile and drop anyway would probably be “low growth” kids.  This allows some recognition of where the kids started from without looking as random as Indiana’s model does, where a kid who got a 525’s growth model looks wildly different from a kid who got a 526; it should be a bit more predictable as well.
• In Year Three, you determine how much they moved from Year Two, and so on.
• Kids who transfer into a district aren’t a problem because they should have some sort of score from their previous district, and even if they were taking a different test in their previous district a percentile score on that test should be trivial to establish.  They then join whatever decile their percentile score belongs to.  If they literally took no standardized tests in their previous district because of their age or their district’s policy on standardized tests, well, the world doesn’t end.
• Teachers and schools are evaluated by how many kids they have in the “average growth” and “high growth” categories.  Those kids should have been enrolled in the district for a certain minimum number of days (I’d say no less than 75% of the school days up to the test week) and– and this may be controversial– should have been present for a certain minimum number of days as well, and I’d say the absence number should be more stringent than the residence number.  I can’t teach a kid who isn’t in school, and I also can’t control whether a kid’s in my classroom or not.  Individual districts or states can determine on their own what their requirements for average growth and high growth numbers should be.

One disadvantage of this is that it does make it more difficult to present school data to the public in an easy-to-understand, useful format.  One big advantage of the pass rate is that parents understand it; moving from 50% pass to 52% pass has a clear meaning, while we’d have to present averages and medians and all sorts of other data to make the new model understandable when we’re comparing schools.  That said, if you want a “one number” comparison, providing the sum of the “high growth” and the “average growth” kids would do nicely; giving all three, combined with averages and medians of actual scores, would provide sufficient information, and anybody who wants to dig deeper (provide numbers per decile, too, maybe) is welcome to.

It’s not great– we’re still paying too much attention to standardized test scores– but it’s certainly better than what we’re doing now.  Feel free to comment (Please!  Comment!) with suggestions and questions.

Be prepared, by the way, for me to find something utterly irrelevant to gripe about tomorrow.

Yesterday I talked about some of the problems that we run into when we try and use standardized test scores as a measure of student, teacher, or school progress.  One of the ideas that always comes up when we talk about this is that we should have “high standards” and “high expectations” for our students, and that when we acknowledge that some people aren’t starting the race at the same place as others we’re somehow not properly Standarding and Expecting things of them.

Sounds good, right?  High is better than low!  Standards and Expecting Things are good, too!  So we should definitely have High Standards and High Expectations.

Well, great.  Sure.

What’s that mean?

No, really.  I’ll wait.

And there’s the problem, see.  You aren’t really saying much of anything useful when you say you have High Expectations and High Standards, and you run the risk of saying something incoherent if you’re not careful.

Let’s address high standards first.  “Standard” has a pretty specific meaning in the education world.  The Common Core is a set of standards; most states still have their own, as the Core isn’t completely phased in yet across much of the country (and is a subject of no small amount of controversy on its own, I should point out).  To say that we should have High Standards is basically just saying that we should be teaching kids material that challenges them to some extent or another and (usually) what the speaker actually means is that kids nowadays should be learning either the exact same stuff they learned in that grade or something more complicated.

You can argue the merits of individual sets of standards; I happen to believe that, at least in sixth grade math, Indiana’s standards are pretty solid, and while I have a quibble or two with how the Common Core handles things I don’t have much of an argument with it.  Just tell me what to teach; I’ll get the job done.  What we don’t have, in any state or jurisdiction or locality that I’m aware of, is any situation where the standards are different for black students or white students or girls or ELL kids or anybody else.  Standards are standards; they’re out there and they’re not disaggregated at all.

So what we’re really talking about here is what we expect from our kids.

And for expectations to be meaningful, they have to be specific.  You can say that you expect everyone to do well.  Great!  Also pointless.  Heck, you can expect anything you want.  I can expect that my students will bring me cookies and milk every day and hurl a virgin sacrifice into the smoking maw of a nearby volcano for me once a month, but that doesn’t mean it’s going to happen.

Lemme say it again: expectations must be specific to specific students in order to be meaningful.  

A few examples, all based on real kids I had last year:  Velma had the highest math score in the fifth grade before landing in my room.  Velma, frankly, doesn’t have a lot of room for growth.  She was already pretty close to getting as high a score as scores can get.

I told her she’d gotten the highest score in the fifth grade.  She was excited and proud.

“I expect the highest score in the school this year, y’know,” I said.  Which, honestly, was asking her for maybe a ten-point bump in her score.  In all seriousness, my challenge with Velma last year was to keep her realistic.  I haven’t seen her ISTEP score for sixth grade yet, because reasons, but when you start off with a score as high as hers is and there’s an upward limit to how well she can do, the simple fact of the matter is no matter how good a teacher I am her score is likely to drop.  Demanding perfection from her would have been stupid.  It would have stressed her out and stressing her out would not have helped her performance.  I used the “best score in the school” line once or twice over the course of the year, mostly to hassle her, but short of a nosedive there’s not much she could have done to disappoint me with her score.  She’s a great kid and I know she did her best.

“Do your best!” sounds like weasel language, doesn’t it?  That’s not high expectations!  Should I have ridden the kid like a donkey all year about driving her enemies before her and hearing the lamentations of their women to avoid the “lacks high expectations” canard?  No, of course not; screw that.  Expecting a kid to do his or her best is really all you need to do if you have a kid who actually will do their best, and there was never any risk of Velma giving me less than 100%.

Fred, on the other hand, had the lowest math score in the fifth grade.  His math score was seriously a hundred points lower than the second-lowest kid I had.  It was so low that I was seriously wondering whether he got off by a number or something and had managed to answer every question on the wrong line.

Early on in the year, Fred’s mother requested a parent/teacher conference with me and asked me flat-out if I thought he was going to pass ISTEP with me as his teacher.  And I told her that there was basically no chance whatsoever of that happening.  I more or less guaranteed her that her son was going to fail again.  He was just too damn far behind; too far to catch him up to grade level in a single year barring a miracle or him moving into my damn house.

And then I told her that I wanted to see a higher point gain from Fred than any other kid I taught this year, and spent about fifteen minutes explaining exactly how we were going to do it, and then worked my ass off all year keeping him as close to on-track as I possibly could.  And now, eight or nine months later, there’s maybe one kid out of the fiftysome I had last year whose score I want to see more than I want to see Fred’s.  I bet I got a 200 point gain out of the kid– not remotely enough to get him to pass, mind you, but about 166% of the score he got last year.

And now there’s Daphne.  Daphne missed passing the ISTEP in fifth grade by something like five points, basically a single question.  I looked her mother in the eye on Open House night, which is before school even starts, and guaranteed her that her kid was going to pass ISTEP this year.  Dumb?  Yeah, probably, but I was in a grand mood.  Daphne spent the year ricocheting from one emotional crisis to another; I caught her cutting herself on two separate occasions (note that I’m the one doing this, not her parents,) rarely turning any work in, etcetera.  Daphne had a terrible year– but a terrible year that had basically nothing to do with her math class or her math teacher.  Do I expect Daphne to have made strides in my math class when Daphne probably did well by keeping herself out of a mental hospital over the course of the year?  Of course not.  I’d like to see that she made some improvement and, honestly, I suspect she did– she didn’t turn in any work, which meant she spent most of the year failing all of my classes (and that’s a discussion for another time) but she seemed pretty on the ball whenever I was assessing her on anything other than turned-in assignments.  But her ISTEP score?  I really don’t care anymore.  Staying the same is just fine.

You tell me: which kid do I have higher expectations for?  Which kid is going to make me look better when and if our test scores get reported solely on pass rates?  And, again, notice: for Fred and Velma, the answer is neither.  One kid failed and will fail again; one kid passed and will pass again.  Neither of them is going to move my numbers at all if we’re using a pass-rate-only evaluation of our test scores.  Daphne is a perfect example of a bubble kid (and, I can’t make this clearer: all three of these kids are real) and so my skills as a teacher and my building’s probation status are going to depend on whether one kid who takes one test on one day spent the night before using erasers to scar her own arms or not.

This is unacceptable.

The pass model fails because it does not encourage high expectations.  It encourages a narrow focus on a narrow band of kids who can be motivated, bribed, pushed or dragged across that line.  And if the state doesn’t particularly like their pass numbers from a test, all they have to do is manipulate the cutscore and– voila!– we had more kids pass than we did last year!  We’re Doing Things over here!  I am certain that Illinois did this while I am teaching there; I had kids pass the ISAT my second year in Chicago who had absolutely no business “passing” anything at all, and the state and CPS crowed and crowed about our pass numbers.  They manipulated the test scores; while I’m not going to go so far as to claim that I didn’t teach my own kids anything, what they did learn from me had precious little to do with their test scores at the end of the year.

I hoped that one of my kids didn’t fall by too much, that one kid failed, and that a third kid just didn’t crater.  And I maintain that all three of those things represent “high expectations.”

Any chance of me convincing anyone of that without 1658 words of explanations?

Tomorrow: A method that I think might actually work.