How a Bush-promoted Christian prison program faked success by massaging data

You don't have to believe in faith-healing to think that an intensive 16-month program, with post-release follow-up, run by deeply caring people might be the occasion for some inmates to turn their lives around. The report seemed to present liberal secularists with an unpleasant choice: Would you rather have people "saved" by Colson, or would you rather have them commit more crimes and go back to prison?

But when you look carefully at the Penn study, it's clear that the program didn't work. The InnerChange participants did somewhat worse than the controls: They were slightly more likely to be rearrested and noticeably more likely (24 percent versus 20 percent) to be reimprisoned. If faith is, as Paul told the Hebrews, the evidence of things not seen, then InnerChange is an opportunity to cultivate faith; we certainly haven't seen any results.

So, how did the Penn study get perverted into evidence that InnerChange worked? Through one of the oldest tricks in the book, one almost guaranteed to make a success of any program: counting the winners and ignoring the losers. The technical term for this in statistics is "selection bias"; program managers know it as "creaming." Harvard public policy professor Anne Piehl, who reviewed the study before it was published, calls this instance of it "cooking the books."