Earlier today Konni Burton, the Republican senator from Fort Worth, took the capitol press corps to task on Twitter for its collective disinterest in the EmpowerTexans/Texans for Fiscal Responsibilty’s biennial “Fiscal Responsibility Index,” which was released yesterday. As is typically the case with that particular ranking, the legislators who earned the highest praise were self-declared “fiscal conservatives,” including Burton herself. She was one of three senators, along with Bob Hall and Van Taylor, to receive a perfect score, and the implication, at least, was that the media had ignored this achievement because it didn’t fit our narrative.

I can’t speak for all media, obviously. But I can explain to readers why I ignored this index. I agreed with some of its conclusions (Burton, for example, is clearly a fiscal conservative, and I thought she was the best of the Senate’s true freshmen.) Still, as I said yesterday, the Fiscal Responsibility Index is too garbled to be meaningful. The methodology is distorted, and—as with all scorecards—overly simplistic. That’s why its results are so erratic. 

Just to make sure I’m clear: It’s true that I don’t respect EmpowerTexans as a group or take most of its “work” seriously. But I’m not dismissing the Fiscal Responsibility Index just because I don’t like Michael Quinn Sullivan or his fiscally subliterate belligerence. I’m dismissing the analysis because it’s not good analysis.

A look at the House rankings will show what I mean. Some of the legislators who are ranked highly on the index are genuine fiscal conservatives, such as Matt Rinaldi, who tried to do away with the state’s funding for film incentives, or Matt Krause, who was one of the first representatives to speak out against spending cap gimmicks. But also appearing in EmpowerTexans’s “top ten” are Molly White, Matt Schaefer, and Scott Turner, all three of whom scored a perfect 100, despite showing no serious interest in fiscal issues this session, much less expertise. Meanwhile, some of the Lege’s strongest fiscal conservatives are clearly underrated in this particular exercise. Mike Schofield, as a freshman, authored a school finance bill that will give the state a chance to appeal an individual judge’s ruling to a panel of three; that’s a pretty big deal and yet he earns a “C” on the Fiscal Responsibility Index, with a score of 76. Giovanni Capriglione authored legislation that will make Texas the first state to have its own gold bullion depository. In addition to being a fiscally sound idea, the measure caused national observers to denounce Texas for being crazy, which is, no pun intended, kind of the gold standard of conservative behavior. And yet Capriglione scored a 72, or a “C-”.

Most ludicrous of all is the Fiscal Responsibility Index’s assessment of Dennis Bonnen. Over the course of the session he a) single-handedly killed the Senate’s effort to exempt property tax relief from counting as “spending”, by announcing that he wouldn’t even ask the members of Ways & Means to vote on such a ridiculous gimmick; 2) saved the state about $1bn this biennium, and every biennium thereafter, by thwarting Dan Patrick’s initial vision for property tax relief; 3) passed legislation cutting the franchise tax rate by 25%; 4) led the House’s border security effort, which yielded a $565m plan; and 5) demonstrated a grasp of the difference between state and local government that eluded all too many of his peers. If anyone deserves to be named a Taxpayer Champion, it’s Bonnen. And yet he, like Capriglione, could only muster 72 points—barely a passing grade.

Such examples help explain why I ignored the 2015 Fiscal Responsibility Index. Again, that’s just my opinion; I can’t speak for all Texas media. An interesting aside is that EmpowerTexans actually rates Jane Nelson slightly more highly than Bonnen; in that respect, at least, they may be more aligned with the “narrative” than I am. But since Burton inspired me to comment I’d like to add a few more thoughts while I’m at it.

The underlying problem with the Fiscal Responsibility Index has to do with methodology. The group that assembled this scorecard is known for its obsessive antipathy to House Speaker Joe Straus, and that factor alone explains some degree of distortion: House members were scored, in part, on whether they voted for Straus’s re-election as speaker. This proved to be an especially erratic litmus test this year. Straus is demonstrably more fiscally responsible than Dan Patrick, and plenty of fiscal conservatives voted for him because—regardless of ideology—he was more qualified to lead the House than Scott Turner, who officially challenged him this year.

Setting that aside, though, the EmpowerTexans scorecards suffers from the same methodological problem that afflicts all scorecards: it’s a scorecard. It has no way to account for anything other than record votes. There’s no weight given to authoring a bill, or negotiating an agreement that enabled the bill’s passage, or building a coalition of support in favor of your bill, even though all of those activities are harder and more meaningful than pushing a button. A scorecard can’t even correctly account for all the votes that are cast. Jonathan Stickland and David Simpson crossed party lines to vote for Cesar Blanco’s DPS amendment during the House’s marathon budget debate, for example. That showed fiscal responsibility on their part–and it was evidence, to me, at least, that even if they sometimes vote in concert with the scorecard, they’re capable of independent thought and activity. And both are, as it happens, highly ranked on this index, but no mention of that vote is made. Instead, Stickland and Simpson are lumped together in a group that includes Krause and Jeff Leach and Rinaldi, but also White and Schaefer and Turner, as if they’re all the same. They’re not.

And so this kind of box-ticking exercise doesn’t do anyone many favors. A well constructed scorecard may be a reasonable proxy for how the legislators behaved. But the Fiscal Responsibility Index? Like I said, it’s too garbled to be meaningful. It wildly underrates some legislators, and wildly overrates others in a predictable and politicized way. The result is that it does no credit to its own honorees, even the ones who happen to deserve it.