Is OPEC a Paper Tiger? A New Study Says Yes

We all know that OPEC colludes to keep oil off the market and prices high. Or do we? There is actually remarkably little agreement on whether OPEC is any good at what it aspires to do. Does membership in OPEC really make countries more likely to constrain their oil output? It’s a question with wide-ranging consequences for everything from the economy to security to climate change.

Jeff Colgan has a forthcoming paper (late draft here) in International Organization (IO), the top journal in the academic international relations world, that answers the big “does OPEC matter” question with a resounding No. The paper is an impressive piece of work that mixes multiple tests of OPEC influence with a careful look into why, if OPEC doesn’t influence oil production, so many people continue to believe otherwise. It should make anyone who is utterly convinced that OPEC is the world oil market’s puppetmaster think twice.

But I think Colgan goes much too far in confidently claiming that OPEC “has little or no impact” on members’ production. This is particularly true if one asks whether policymakers and market participants, rather than only political scientists, should conclude from his paper that OPEC doesn’t matter. As a corollary, I think he ultimately goes too far in trying to explain why so many people believe in a “myth” – what they believe, at least in its subtler forms, may not actually be a myth at all.

Colgan applies four tests to his data; I find the first three largely underwhelming and the fourth far more interesting. The first question he asks is whether entry into OPEC reduces a country’s oil production (relative to trend) and whether exit from OPEC increases it; he finds that OPEC has no detectible influence. But it’s unclear that the counterfactual here is correct. It’s just as plausible that states that enter OPEC are ones that would otherwise see accelerating production (and vice-versa for exits) – after all, being an up-and-coming oil producer is presumably what makes you think about joining OPEC, and being on the decline makes you consider leaving. Colgan’s methodology is borrowed from studies of WTO entry and trade, but it’s not clear that the situations are analogous, since already-accelerating trade presumably isn’t a typical motivation for WTO entry.

The second and third tests look at OPEC quotas. One shows that OPEC countries consistently cheat; Colgan acknowledges that this is a weak test given that quotas can be set at levels that anticipate cheating. The other is more persuasive: it shows that OPEC quotas have very little predictive power when it comes to explaining month-to-month variations in OPEC countries’ production.

I should pause here for a moment to be clear about what Colgan isn’t claiming. He isn’t claiming that individual OPEC  members (notably Saudi Arabia) don’t exercise market power in ways that prop up the price of oil. What he’s claiming is that to the extent that happens, it’s about individual country decisions, and OPEC membership has nothing to do with it. That’s the right test when you’re asking whether OPEC is an effective cartel.

So back to the tests – because the fourth one is the most interesting. Colgan notes, correctly, that OPEC members may restrain production not through short-term quotas but through long-term restraint on oil production investment. This is a popular view. (Some also believe that the OPEC core restrains production through a form of price coordination, a possibility that Colgan briefly raises and then rejects, I think too readily.) And indeed Colgan shows that OPEC membership is a decent predictor of low oil depletion rates. But he then claims to show that this is an artifact: if one allows depletion rates to be influenced by fiscal strength (measured essentially by oil reserves per capita), investment risk, and several other factors along with OPEC membership, one finds that fiscal strength and investment risk help explain depletion rates, and that OPEC membership no longer matters.

This is interesting stuff and should make people think hard about whether OPEC has real influence on its members’ behavior. But I’m extremely wary of the strong conclusion for a couple big reasons. First, it’s far from clear that OPEC membership itself doesn’t influence countries’ investment climate. To be certain, leaders don’t wake up and say “I’m an OPEC member – I should have an awful investment climate!”. Attitudes toward investment are, instead, shaped by a much broader range of factors. But having decided to underinvest in oil production, a state is less likely to feel pressure to improve its investment climate, since it has less need to. To be clear, I have no idea whether OPEC membership influences decisions regarding oil production, and hence investment climates, in this way. But Colgan’s methods wouldn’t detect that even if it did.

My other big worry has to do with how the paper – following standard political science practice – handles statistical significance. Political scientists search for “truths”. When Colgan says that OPEC has “little or no impact” on members’ production levels, what he means is that he can’t be at least 90 percent certain that OPEC membership influences member countries’ production levels. (More precisely, as best I can tell, he can’t reject with 90 percent or greater confidence the hypothesis that OPEC membership doesn’t influence members’ production levels, so he accepts it.)

This may be the right threshold for political science but it’s a dangerous one for policymaking. For example, it turns out that one of Colgan’s tests (#3 in Table 2) allows one to conclude with 75 percent confidence that OPEC membership influences member countries’ production levels (again with the same caveats about double negatives as above). Another test (#4 in Table 2) shows that even if one separates out Saudi Arabia, one can still conclude with 75 percent confidence that OPEC membership influences remaining member countries’ production. And a third test (#5 in Table 2) shows that if one further separates non-Saudi OPEC members into a “core” and a “periphery” one can still conclude with 70 percent confidence that they’re all influenced by OPEC. (One note of caution here: Seventy percent confidence isn’t nothing, but it isn’t as much at one might intuitively think; fifty percent confidence would mean that we have no clue what OPEC influence is. The way one interprets these sorts of numbers should depend strongly on one’s priors, i.e. other pre-existing beliefs about OPEC.)

Should a policymaker or market participant write off OPEC because data analysis suggests that there’s only a 75 percent chance (rather than a 90 percent one) that it’s influential? (Or, put a clumsier but slightly more accurate way, should they ignore OPEC because a statistical test can’t conclude with 90 percent confidence that people who say “OPEC doesn’t matter” are wrong?) Of course not. He or she should discount the influence of OPEC in his or her assessment of the costs and benefits of any potential action, given uncertainty about whether OPEC is genuinely influential, but it would be awfully risky to proceed on the assumption that OPEC has no influence at all.

In fairness to Colgan, he addresses this indirectly, writing that “it is impossible to affirm the null hypothesis (i.e. to prove that OPEC has no impact)”, and then estimating the impact that OPEC may have had on oil production. He uses several factors (not including OPEC membership) to predict OPEC oil production and then estimates that non-Saudi members under-produced by 6.6 percent relative to that (equivalent to 1.6 million barrels per day in 2009). Colgan asserts that it is “difficult to believe that such an amount is having a major impact on world oil prices”. But there remain three issues here: 1.6 million barrels a day is not a trivial amount of oil; Colgan does his projections based on a data set extending back to 1980 even though analysts typically believe that OPEC restraint based on under-investment is a far more recent phenomenon (Colgan subdivides his data set into shorter time periods elsewhere in the paper but not here); and the estimate of a 6.6 percent percent shortfall presumably has large uncertainties that might mean that OPEC impact is considerably greater.

Bottom line? Colgan’s paper should deflate some of the hyper-confident claims that OPEC rigs the world oil market. I hope it sparks some constructive debate. It would be awfully unwise, though, for policymakers or market participants to quickly flip to an equally over-confident belief that OPEC doesn’t matter.