-- John Kemp is a Reuters market analyst. The views expressed are his own --
By John Kemp
LONDON (Reuters) - Stanford University's Professor Kenneth Singleton has mounted the most wide-ranging and influential assault so far on the orthodoxy among policymakers and academics that speculation does not affect commodity prices.
Singleton's paper on "Investor Flows and the 2008 Boom/Bust in Oil Prices" is generating waves across the commodity world because he is a highly respected econometrician whose findings and arguments cannot be easily dismissed as the work of the usual cranks and opponents of free markets (www.stanford.edu/~kenneths/OilPub.pdf).
Singleton argues existing theory fails to acknowledge the effect of market participants trying to anticipate the behaviour of competitors. Once this component of trader behaviour is included, the view that futures prices are determined solely by economic fundamentals is undermined.
But Singleton is just the most prominent of a new wave of economists working on commodities who are modifying long-held views about how physical and derivative markets interact.
Singleton builds on the work of Ke Tang (Renmin University), Wei Xiong (Princeton), Bahattin Buyuksahin (IEA), Michel Robe (American University) and Yiqun Mou (Columbia Business School) who have investigated links between investment and prices and challenged or modified the previous consensus.
Singleton's findings are consistent with a presentation on "Crude Oil Price Formation" given by EIA Administrator Richard Newell in February 2011, and now showcased in a new section on the agency's website, which gives financial factors a bigger role explaining commodity price movements.
It all stands in marked contrast to the orthodoxy of a previous cohort of economists led by Scott Irwin (University of Illinois), Craig Pirrong (University of Houston) and Jeffrey Harris (former chief economist of the U.S. Commodity Futures Trading Commission) inspired by the work of Holbrook Working at Stanford's Food Research Institute in the 1950s and 1960s.
It was this establishment that influenced the famous CFTC Staff Study in July 2008 which found "current oil prices and the increase in oil prices between January 2003 and June 2008 are largely due to fundamental supply and demand factors".
It shaped the conclusions of ISOCO's Task Force on Commodity Futures Markets, led by the CFTC and Britain's Financial Services Authority: "economic fundamentals, rather than speculative activity, are a plausible explanation for recent price changes in commodities".
And the old orthodoxy was officially endorsed when Irwin was selected to write a study on "The Impact of Index and Swap Funds on Commodity Futures Markets" for the OECD's Food, Agriculture and Fisheries division. Irwin found: "Based on new data and empirical analysis, the study finds that index funds did not cause a bubble in commodity futures prices."
Differences between the 2008 CFTC staff study (which was unequivocal that there was no evidence that speculative factors had influenced prices) and Newell's 2011 EIA presentation (which admitted "researchers are finding some evidence suggesting the price run up and decline may have been exacerbated by the formation and collapse of an oil price bubble") shows how far the debate has evolved.
Orthodox views still predominate but are under increasing pressure from a new wave of revisionists open to the idea financial flows could disturb prices, at least temporarily.
It is easy to forget that before the boom started in 2004, commodities was a niche field characterised by falling real prices and not much money or excitement. It was dominated by a handful of experts and the main theories of price determination had been settled for more than 50 years (John Maynard Keynes and Harold Hotelling in the 1930s, Working in the 1950s and 1960s).
But surging prices have made the sector lucrative and interesting, and drawn in a far greater number of researchers as well as investors. Newcomers have less intellectual and emotional attachment to the old consensus.
Importantly, the repackaging of commodities as an "asset class" and the dramatic rise in investment via passive commodity indices and actively managed funds may have altered existing relationships in ways that invalidate the old theories or at least change the way they are applied.
Orthodox views insist there is "no evidence" speculation has influenced prices and are grounded in approaches that stress rational expectations and efficient markets. They are thus open to challenge on both empirical grounds and theoretical ones. Singleton and others in the new wave have attacked them on both.
On the empirical side, absence of evidence is not the same thing as evidence of absence. Lack of statistically proven relationships between commodity prices and speculative or index positions could be due to shortcomings in the data used or the econometric techniques employed.
Published work has been based on data taken from CFTC's commitments of traders' (COT) reports in either public or confidential versions. But according to Singleton "the CFTC's COT data does not give a reliable picture of the overall demand for and supply of commodity risk exposure."
Singleton cites a raft of credible studies showing heightened investor participation may have affected various elements of price formation.
It may have increased correlations among forward prices, as well as with equity markets, especially for those commodities included in indices, with spillovers to commodities that were not included in any index and do not have traded futures.
The empirical attack is twinned with a theoretical challenge. Singleton takes aim at rational expectations and efficient market approaches that rely on "representative" investors, producers and consumers sharing the same public information and interpretation of it, and differing only in the private information they possess.
Under such assumptions it is difficult to generate the volume of trade actually observed in commodity markets. It is more realistic to acknowledge they disagree about the interpretation of even commonly available information.
Once there are differences of opinion "behaviour in the spirit of Keynes's beauty contest may arise naturally. It is typically optimal for each participant to forecast the forecasts of others ... Participants will try to guess what other participants are thinking and to adjust their investment strategies accordingly," according to Singleton.
"Groups of traders that hold different views will naturally engage in speculative activity with each other ... this heterogeneity leads investors to overweight public opinion and this in turn exacerbates volatility". Once traders start to factor in each other, behavioural approaches (including the influence of herding and crowding) become much more important.
Singleton also highlights Bayesian feedbacks in which uncertain investors use prices to deduced fundamentals rather than the other way around (ie prices are rising so demand must be strong) resulting in boom/bust cycles.
Investors do not need to have private information for their actions to impact on commodity prices. Differences of opinion about public data are sufficient to generate boom/bust cycles.
Co-movements in prices for commodities insides and outside indices, with and without associated futures contracts, need not invalidate arguments about the effects of speculation. "Participants in all commodity markets should find it optional to conditions on prices in other markets when drawing inferences about future spot prices".
Finally "the fact that investors are learning about both fundamentals and what other investors know or believe about future commodity prices may mean that the release of a seemingly small amount of new information about supply or demand has large effects on prices".
Singleton has not proved his theory about the speculative impact of investment flows on commodity prices beyond doubt. But his paper offers a richer and more realistic view of how real markets operate than the rather stylised formulations that have been popular with the old guard.
More importantly, Singleton and the others have cracked a stifling orthodoxy and opened room for a genuine discussion about how commodity prices are really formed.
Editing by Anthony Barker