Just as 2020 was the year everyone turned into self-proclaimed epidemiologists, 2022 seems to become the year of the energy market experts, as energy prices go through the roof and words like “merit order curve” or “marginal cost dispatch” become layman’s terms. While alarm bells started to ring within the energy community already before Putin decided to throw a fit (sorry for the sligh understatement), the harsh reality we (or at least our wallets) were about to face became quite clear by the time electricity bills started to show up with a few too many 0s to be funny.
Before long, proposals for drastic political measures and market design overhaul measures to protect consumers against the Merciless Market were popping up on my newsfeed, giving me quite a bit of food for thought over the past few months. OK, maybe I should also get out more, but let’s keep this space for nerding around. To understand a bit better what this “market mess” is all about, I invite you to take a little stroll down History lane, to explain where we came from, why we took the decisions we did, and how the current situation might call for a much needed update of the way we sell, buy, store, and transport a good without which very few things in our modern society would function: electricity. For the sake of your time (and to respect the boundaries of my limited expertise), I will focus the discussion on the electricity wholesale spot market (don’t worry about the technical linguo for now, just replace it with the “power market” you usually read about in the news), and restrict it to the European context, although the Old Continent is far from being an exception.
In the “Trente Glorieuses” (Glorious Thirties) decades characterising post-war European economy, every progression metric seems to follow an exponential curve, boosted by reconstruction plans (nothing like a good ol’ war to get people back to work), modern industrialisation, and groovy Elvis Presley bangers. Fuelling this growth required an equivalent growth in electricity production, and states were frantically building new coal-fired power plants across countries to keep up with the demand.
Evolution of British power generation mix from the roaring 20s to the less roaring last decade (adapted from [1])
At the time, planning, construction and operation of the electricity sector was a state monopoly, characterised by few but very large-scale power plants spread across the country, in true central-planning style. All these power plants were eventually connected with each other through a transmission grid for redundancy and resiliency purposes, essentially turning the power system into one big machine operating on a country scale (see previous post for further details).
Coal was still the main fuel used for most plants, but coal production was rapidly declining across Europe. Different alternative options were explored, some countries going for oil-fuelled power plants (in 1972, oil represented 81% of Danish power and heat production!), while others invested massively in nuclear expansion (I’m looking at you, France). But all in all, the idea remained the same: burn whatever you can to generate steam, pass it through a turbine, and dispatch the generated electricity to whoever needs it. Both production and consumption prices were fixed by the state and updated once in a while based on fuel prices. The daily operation schedule of this big power system was then only a matter of optimising the relative production volume of each generator based on consumption patterns. The people want more, burn more. The people want less, burn less.
The first fossil-fuel peaks of many more to come (taken from [2])
Unfortunately, things were about to get more complicated. With the oil crises of 1973 and 1979, many Europeans woke up from the party with a slight hangover, as words such as “rationing” were making the frontlines again. The answer at the time was energy efficiency: if energy costs more, let’s use it in a smarter way. Until then, central planners’ main priority had been about keeping up with ramping demand to keep the lights on, and not necessarily to do so in a cost-effective way.
It turns out that when we talk about energy efficiency, there was a new kid on the block which definitely was getting some attention: natural gas. The higher energy density of natural gas allows to reach higher combustion temperatures compared to coal, which in turn allows to transform a larger part of this heat into electricity (some call it black magic, others call it thermodynamics). Additionally, the exhaust heat in a gas turbine is hot enough to be reused in a second turbine, a process called Combined Cycle Gas Turbines (CCGT), further transforming heat into electricity. All in all, this makes natural CCGT achieve a thermal efficiency (share of heat released by fossil fuel combustion converted into electricity) of around 60%, compared to the bleak 35% of single-cycle coal-fired steam turbines. To put it short, you get more bang for your buck.
Coincidentally (if you do believe in coincidence in this sort of context), the 70s also corresponds to the time where major natural gas reserves are found in the North Sea. The United Kingdom, Norway, Denmark, the Netherlands suddenly find themselves with large home-grown energy reserves just waiting to be burnt (I mean, what else would you do with natural resources, apart from burning them up?!). To put the final nail in King Coal’s coffin, and get a few environmentalists on board, it had to be underlined that natural gas considerably reduces SO2 and NOX emissions, which are linked to acid rains, and reduces particle emissions due to “cleaner” combustion at higher temperatures.
The only question that remained to be answered was how to finance this fuel transition and how to justify shutting down these stranded coal-generation assets.
This is where the ideological background of the times played a major role. Market liberalisation, a movement embodied by the fierceless Thatcher-Reagan duo, was promoting the concept of fair competition between independent actors to achieve the most efficient outcome, which was to be understood as the most price-competitive outcome. The result of this endless competition, called the market, was to be reflected by market prices. Fundamentally, a major difference between a centrally planned and a liberalised system lies in information access for decision making. In a centrally planned system, the central regulator takes decisions on behalf of each participants, based on the information it has access to. In a well-designed liberalised market, market prices reflect the aggregated decisions made by independent market participants, where each participant takes action based on its own information, assuming each individual is best informed about its own state. Market prices should therefore reflect in real-time the state of the electricity system, without requiring constant monitoring and supervision of the state in a Big Brother-like fashion. High prices would then encourage the entry of new more efficient participants, quicking out the unproductive incumbents, and the whole system would manage itself efficiently on its own. Easy peasy.
Despite the many assumptions taken to make such a liberalised economy work properly, one must admit the difficulties for centralised planners to keep everything under control and operate smoothly, especially as the system complexifies. An abstract from an investigation undertaken by the Monopolies and Mergers Commission under Thatcher gives a good idea of politicians’ impressions of the state-controlled coal industry at the times:
“excess output, keeping [coal] pits open and building too many stations; financial laxity in planning and project execution; prices set at artificially low levels relative to costs; and labour bias and overstaffing .[…] By the early 1980s, the nationalised industries had built up sufficient inefficiencies to warrant more radical surgery.” [ref]
An interesting modern-day case study is the power crisis faced by China in late 2021, during which coal power producers had to stop power generation for several weeks in a row to avoid operating at a loss. These generators’ revenues were fixed by state-regulated power prices determined a year in advance, while their operational costs depended on the fluctuating prices of coal, which was traded as a global commodity and experiencing post-COVID recovery record highs. In the regulated context in which these producers were evolving, they were not able to reflect this increase in operational costs on the price at which they were selling their electricity. Interestingly enough, this (among many other events) sparked a whole series of market reforms in China to encourage a more market-based operation of its power system.
Back in Europe in the late 80s, the inefficiency of the power sector became increasingly apparent with the slowdown of economic growth, the push for energy efficiency after the oil crises, and the progressive industrial offshoring of some of the largest power consumers from the early 90s, leaving many generators to operate below their full potential. Natural gas power plants, in addition to being more efficient, were also smaller and less capital intensive, allowing private companies to invest in them more easily. Under these circumstances, the red carpet seemed already rolled out for electricity market liberalisation. Spearheaded by the Brits riding the wave of Thatcherism, closely followed by their Norwegian neighbours, European countries started liberalising their power sector one after the other, from the mid 90s onwards. Going back to our first graph in this post, this corresponds to the time where CCGT power production (in red) is truly picking up.
Initially, each country was cooking up its own recipe, making rules based on their own generation mix. But by and large, most countries still relied on large controllable generators (called dispatchable generators in the jargon, meaning their production can be ramped up and down depending on needs), be it coal, natural gas, nuclear or hydropower. These generators were put in competition with each other to satisfy consumer needs at the cheapest price. Who was allowed to produce how much at what time needed to be decided close enough to real-time consumption to minimise consumption forecast errors, but far enough in advance to give enough time for operational rescheduling. Operational schedules also had to avoid too large variations within the day, as turning a generator on and off always takes a bit of time, but had to be accurate enough to follow daily consumption variations. Most countries therefore settled for day-ahead volume clearing with an hourly time step resolution (this is getting technical here, but bear with me). Regarding pricing, the frontrunners of market liberalisation were often transitioning from a coal-based to a gas-based system. Unlike nuclear or hydropower, the operational costs (OPEX) of natural gas plants represent a substantial share of total costs over the lifetime of the power plant, due to fuel costs. Gas power plant investors would therefore firstly be worried about recovering these variable costs.
Cost distribution of different power generation technologies (from [3])
In that context, the concept of marginal pricing became a self-evident solution: ranking the generator’s production bids from least to most expensive, production bids are added until the required consumption level is matched, at which point the clearing volume is reached, which corresponds to consumers’ total demand. This clearing volume is considered non-negotiable: if Bob wants to turn on his jacuzzi on Christmas eve while everyone else has their turckey in the oven, then so be it. And at the right temperature, please! In the jargon, we call that inflexible loads. In reality, you can call that being selfish, or just blissfully ignorant (let’s face it, we’re all Bobs when it comes to switching on the light).
The price of the final bid required to meet this clearing volume is the clearing price, which all generators included in the clearing volume get paid. Since all bids meeting the clearing volume are ranked in increasing price order, generators operating at clearing price cover their operational costs, while generators operating below clearing price receive profits, which they can use to cover their initial capital investments (CAPEX in the linguo).
The infamous marginal cost diagram, adapted from [4] with added comments from yours truly
While this sounds all good in theory, it implies that long-term investment projects depend on daily market price variations for their return on investment. In a world of stable and cheap fossil fuel supply, this could be an acceptable risk to take. But if the operational costs of the price-setting power plant start to go on roller-coaster rides as fuel prices become more unstable, such long-term investments become risky to say the least. Nuclear technology, a classic example of high CAPEX-low OPEX technology requiring stable long-term revenues, is typically not a great fan of playing yoyo with market prices. No wonder the French and their centrally planned nuclear fleet were dragging their feet when it came to liberalising their market. But I guess having just won the football World Cup in 98, they needed something to complain about to stay true to themselves, and eventually even France jumped on the liberalisation bandwagon.
Anyone having followed the news in the recent months will have many objections to this idealised vision of a market which I just presented to you. That is because, well yes, sometimes it can hurt when expectations hit reality. A part of this reality is a little molecule, which initially was just seen as a threat to polar bears and condescendingly called an “externality” by economists, but which quickly came to become a cornerstone of energy policy and ruined the neatly planned-out power market reforms of the 90s. Stay tuned for the next episode, where carbon emissions get introduced into the equation, and where the market structure based on over-capacity and dispatchable power plants starts to get challenged by a changing reality.
References and further reading for this article:
-Glachant et al. (2021) Handbook of Electricity Markets https://www.e-elgar.com/shop/gbp/handbook-on-electricity-markets-9781788979948.html
-Stagnaro (2015) Power Cut? How the EU Is Pulling the Plug on Electricity Markets https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3893649
-IRENA (2022) Potential Limitations of Marginal Pricing for a Power System Based on Renewables https://www.irena.org/Technical-Papers/Potential-Limitations-of-Marginal-Pricing-for-a-Power-System-Based-on-Renewables
-Jamasb and Pollitt (2005) Electricity Market Reform in the European Union: Review of Progress towards Liberalization & Integration, Energy Journal