The right way to defend (or attack) the Congressional Budget Office
The Congressional Budget Office is in the crosshairs for its prediction that the republican health care bills will cause 22 million people to lose health insurance. Eight of the organization’s former leaders wrote a letter to defend it. Who’s right, the CBO’s critics or its defenders?
Congress created the CBO in 1974 as an independent, non-partisan group of analysts separate from the executive branch. From the CBO website:
CBO is strictly nonpartisan; conducts objective, impartial analysis; and hires its employees solely on the basis of professional competence without regard to political affiliation. CBO does not make policy recommendations, and each report and cost estimate summarizes the methodology underlying the analysis. Learn more about CBO’s commitment to objectivity and transparency.
Despite this mission, the CBO’s sometimes unpopular estimates open it up to criticism. House Speaker Paul Ryan call its analyses of health care coverage “bogus.” Health and Human Services Secretary Tom Price said “The CBO was wrong when they analyzed ObamaCare’s effect on cost and coverage, and they are wrong again.” And Tom Garrett, a Republican congressman, accused the CBO of conspiring against Congressional Republicans.
Former CBO directors defend the organization
Former leaders of the CBO, appointed by both Democrats and Republicans, wrote an open letter to congressional leaders and posted it on Medium. Let’s take a look. I’ve highlighted weasel words and added commentary.
Dear Mr. Speaker, Madam Leader, Mr. Majority Leader, and Mr. Minority Leader:
The undersigned represent every former Director of the Congressional Budget Office (CBO). We write to express our strong objection to recent attacks on the integrity and professionalism of the agency and on the agency’s role in the legislative process.
Commentary: It’s impressive that all the former directors are unified. But I’m not sure what the difference between a regular objection and a strong objection is.
CBO began serving the Congress in 1975. Over the past 42 years CBO has been firmly committed to providing nonpartisan and high-quality analysis — and that commitment remains as strong and effective today as it has been in the past. Because CBO works for the Congress, and only the Congress, the agency’s analysis addresses the unique needs of legislators.
Commentary: It is difficult for nonpartisan analytical people to defend their methods (I should know, since I was one). The CBO must stress its nonpartisan role. The analysis of the CBO is thorough, comprehensive, model-based, and doubtless reflects the latest and most accurate econometric and analytical methods. These would be much better descriptors than “high-quality,” which is strangely imprecise. In any case, what’s in question is not the quality of the analysis, but its accuracy.
To meet the standard of nonpartisan objectivity, CBO makes no recommendations about policy, regularly consults with researchers and practitioners with a wide range of views (as can be seen in the agency’s panels of advisers and reviewers for major studies), and enhances its transparency by releasing extensive descriptions of its analytic techniques and forecast record. To produce estimates of high quality, CBO uses its detailed understanding of federal programs and economic conditions, ongoing interactions with government officials and private-sector experts, the best academic research, and the latest available data consistent with the timing of the Congressional budget process.
Commentary: These details support the credibility of the work and its nonpartisan nature. In my opinion, though, the pileup of weasel words (“regularly”, “high-quality,” “detailed,” “best”) undermines the impact of this paragraph. Given that these are quantitatively skilled economists, this description is surprisingly qualitative.
CBO’s approach produces consistent comparisons of competing legislative proposals and unbiased projections of the impact of policy changes. Unfortunately, even nonpartisan and high-quality analysis cannot always generate accurate estimates. Policy changes are often complex, the economy is dynamic and defies precise prediction, and many policies are modified over time. However, such analysis does generate estimates that are more accurate, on average, than estimates or guesses by people who are not objective and not as well informed as CBO’s analysts.
Commentary: This is the key statement about methods and modeling. It’s clear and describes the potential sources of inaccuracy. It belongs at the top.
In sum, relying on CBO’s estimates in the legislative process has served the Congress — and the American people — very well during the past four decades. As the House and Senate consider potential policy changes this year and in the years ahead, we urge you to maintain and respect the Congress’s decades-long reliance on CBO’s estimates in developing and scoring bills.
Commentary: “We’re the best you’ve got.”
An analyst’s view of agencies like the CBO
As an analyst, I’ve published many forecasts and projections. A lot of them were wrong. As Yogi Berra supposedly said, “It’s rough to make predictions, especially about the future.” So I understand where the CBO is coming from.
The CBO overestimated the number of people who would be insured under the Affordable Care Act (see the Washington Post graphic, below). But as the Post explained, much of that overestimate was a result of the Supreme Court ruling that allowed states not to expand Medicaid — and the subsequent decision by 18 states not to do so. The other factor that caused their models to be off was that more people than they expected preferred to pay the penalty for lack of insurance rather than pay for insurance. That’s a tough thing to predict, because neither the insurance marketplaces nor the penalty existed at the time the CBO was making predictions.
Does this make the CBO projections worthless? Far from it. Does it make them accurate? Not quite. That’s why the letter describes them as “high-quality” forecasts — they’re the best available way to prediction an inherently uncertain future.
Rather than criticizing the CBO as partisan (it clearly isn’t), an honest critique would ask “What assumptions does the CBO make? What parts of its model are subject to change based on future events? How would those events affect the model? And what has the CBO gotten wrong in the past, and how did it make adjustments for those mistakes?” These are debates that analysts and economists can participate in, and they shed light rather than just generating heat.
Whenever you look at a projection, seek out bias first. Then examine how the projection is constructed and what assumptions it relies on. That’s the intelligent person’s approach to a projection. I wish more politicians would adopt it.
