Expecting the unexpected: Why the oil price keeps surprising us

Christiane Baumeister, Lutz Kilian 08 February 2016



As of late January 2016, the West Texas Intermediate price of crude oil is approaching $30, down from $106 in June 2014. It is fair to say that few observers expected such a major decline in the price of oil as of June 2014. For example, the IMF in October 2014 predicted that the "average price of oil will be … $99 a barrel in 2015 and will remain unchanged in real terms over the medium term” (IMF 2014). The IMF was not alone in its assessment. This is not the first time either that oil price expectations missed their target spectacularly. The last 40 years of oil price fluctuations are replete with episodes of unexpected changes in the price of oil (see Figure 1). In a recent discussion paper, we explore the question of how economic agents form expectations about the price of oil and of why future realisations of the price of oil often differ from these expectations (Baumeister and Kilian 2016).

Figure 1. The real price of West Texas Intermediate (WTI) crude oil

Not surprisingly, there is a range of different approaches to quantifying oil price expectations. For example, is has been shown that consumers tend to expect the price of oil to grow from its current value at the expected rate of inflation (Anderson et al. 2011). In contrast, many central banks and international organizations simply equate oil price expectations with the prices of oil futures contracts (IMF

2005). More sophisticated measures of the market expectation of the price of crude oil may be obtained by adjusting the futures prices to account for the risk premium in oil futures markets (Hamilton and Wu 2014). Finally, economists often generate oil price expectations based on models of the global market for crude oil that incorporate data on past oil prices, on the global business cycle, on changes in oil inventories and on global oil production (Kilian and Murphy 2014).

Why is there no agreement on the expectation of the price of oil?

These four measures of oil price expectations may differ substantially in practice, especially during episodes of major oil price fluctuations. The reason is that each measure is based on a different set of information. For example, expectations generated based on estimates of oil market models focus on the predictive content of a small number of carefully chosen indicators of the state of the global oil market including variables that are known to contain forward-looking information such as changes in oil inventories.

Financial market expectations take account of these economic determinants of the price of oil, but they also embody other determinants such as the market’s assessment of the political stability of the Middle East, of the prospects for new oil discoveries, or of the profitability of new oil extraction technologies, all of which are difficult to capture by conventional regression-based methods. This does not necessarily mean that financial market expectations are more accurate, however, because financial market participants are also prone to misinterpreting evidence or to overreacting to news.

In contrast, consumer expectations abstract from most of the information considered useful by many financial market participants and economists. Because the interpretation and predictive content of such information often is questionable, this alternative strategy is not without potential merit either.

Which measure of oil price expectations is most accurate?

It may seem that the best measure of oil price expectations would be the one which comes closest on average to predicting the future price of oil. Indeed, it can be shown that there is a clear statistical ranking among these expectation measures. Economists’ oil price expectations tend to be more accurate than financial market expectations, which in turn are more accurate than policymakers’ expectations based on the oil futures price; the least accurate measure by far being consumers’ oil price expectations. This ranking is consistent with the view that understanding the economic determinants of the price of oil helps in forming accurate expectations about the evolution of the price of oil.

Why do we continue to be surprised by oil price fluctuations?

Even the most accurate expectations measure, however, is no safeguard against occasional large unexpected movements in the price of oil, more commonly referred to as oil price shocks. The reason is quite simple. Even if one has a good understanding of the past evolution of the global oil market, the price of oil will only be as predictable as its underlying determinants, which include, for example, the expected strength of the Chinese economy, the political stability of the Saudi monarchy, and the extent to which US shale oil producers will have access to external financing. Unless we can accurately foresee the future evolution of determinants such as these, surprise changes in the price of oil driven by unexpected shifts in oil demand or oil supply will be inevitable.

This problem arises whether one formally models the key determinants of the price of oil, as in vector autoregressive models of the oil market, or relies on an intuitive understanding of the oil market, as financial market participants are prone to. In our paper we examine in some detail, drawing on specific historical episodes, why anticipating future shifts in the demand for and supply of oil has remained difficult, even as the market’s understanding and economists’ understanding of the oil market has improved.

Is predictive accuracy the relevant criterion for judging oil price expectations?

It is easy to lose sight of the fact that questions of statistical accuracy are only one criterion for choosing between oil price expectations. We make the case that the relevant question for understanding the effects of oil price shocks on oil-importing economies may not be so much which of these expectations measures is most accurate in predicting the price of oil, but whose expectations are relevant for economic decisions. Hence, all four oil price expectations measures we study are relevant to economic analysis, and the notion that there is a unique measure of the oil price shocks experienced by an economy that can be measured by statistical methods may be quite misleading.

For example, if one wants to understand which car a consumer will buy (or whether the consumer will choose to buy a car at all), then the consumer’s own oil price expectations, no matter how inadequate, may be more relevant than the most accurate oil price expectation that economists can construct. It is the consumer’s expectation that matters for understanding the effects of the decline in the price of oil since June 2014 on automobile purchases and, more generally, on the economy, as consumers adjust their spending.

The divergence of consumers’ oil price expectations from objectively more accurate alternative expectations measures does not mean that consumers are irrational. After all, consumers cannot be expected to have the time and resources to become experts in oil price prediction themselves or to make sense of the wide range of competing oil price predictions reported in the media.

We provide evidence that consumers were surprised by the evolution of the price of oil to a greater extent than oil economists, for example. Likewise, policymakers’ measures of oil price expectations may not line up well with consumers’ oil price expectations, complicating the design of appropriate policy responses. These examples suggest that we need to rethink how the effects of oil price shocks are modelled in macroeconomics.

Concluding remarks

Macroeconomic models of the transmission of oil price shocks to date have not allowed for heterogeneous oil price expectations. The standard approach has been to postulate that economic agents in the model form expectations consistent with the structure of the economic model with no allowance for differences in expectations across consumers and financial markets, for example, or for the use of information not contained in the model.

Given the differences in the timing and magnitude of oil price shocks, depending on which expectations measure is used in computing these shocks, it is central to allow for heterogeneous expectations in modelling the effects of oil price shocks. Such distinctions are likely to be of first-order importance for applied work and for economic policy.


Anderson, S T, Kellogg, R, Sallee, J M, and R T Curtin (2011), “Forecasting Gasoline Prices Using Consumer Surveys,” American Economic Review: Papers & Proceedings 101: 110-114.

Baumeister, C and L Kilian (2016), “Forty Years of Oil Price Fluctuations: Why the Price of Oil May Still Surprise Us,” CEPR Discussion Paper 11035.

Hamilton, J D, and J C Wu (2014), “Risk Premia in Crude Oil Futures Prices,” Journal of International Money and Finance 42: 9-37.

International Monetary Fund (2005), World Economic Outlook, Washington, DC.

International Monetary Fund (2014), World Economic Outlook, Washington, DC.

Kilian, L, and D P Murphy (2014), “The Role of Inventories and Speculative Trading in the Global Market for Crude Oil,” Journal of Applied Econometrics 29: 454-478.



Topics:  Macroeconomic policy

Tags:  oil, oil price, China, price expectations, prices

Assistant Professor of Economics, University of Notre Dame; Research Affiliate, CEPR

Professor of Economics, University of Michigan; and Research Fellow at CEPR.

Vox Talks