The Decarbonised Electricity System of the Future: The ‘Two Market’ Approach

Abstract

This paper explains why current electricity markets are not fit for purpose and proposes a new consumer-driven two-market design to support very high penetrations of intermittent renewable energy. The structure encourages demand flexibility; enables consumers to choose what mix of energy they wish to purchase from the two markets (as available renewable energy or on demand flexible energy); encourages them to shift their demand to the as available market (and to acquire storage and other load-shifting appliances); provides a market-basis for financing flexible plant; and offers an exit strategy for government subsidies to renewables and flexible plants

Overall concept

This Part introduces the ‘two market’ design which the authors have proposed for the decarbonised electricity system of the future and sets out the main principles underlying the design. Part 2 below expands the analysis to show in more detail how the design tackles the various challenges related to the decarbonisation of the system.

The starting point is that current energy-only electricity markets are broken (as explained in more detail in the references), especially in markets with high penetration of intermittent renewables. Energy-only markets are designed to discriminate between sources with different short run marginal costs (srmc); by selecting the lowest cost plants they should ensure both short and long run efficiency. But this design is based on the assumption of dispatchable plants with varying marginal costs, an assumption which will no longer hold good in the decarbonised market of the future if, as seems inevitable, it is dominated by intermittent plants with low or zero srmc. In such circumstances energy-only markets cannot remunerate investment, and may not be able to provide effective signals for operation or for consumers. Furthermore, there is no exit strategy – as long as plants with near-zero srmc dominate, they need support from outside the wholesale market, but their presence in the market creates ‘pecuniary externalities’ (OECD/NEA Paris 2012, pp 34-37) which distort that market and lead to a need for support for conventional plants via capacity payments and the like. In this situation there are no market signals to optimise the system – the quantity and type of renewable plants are determined by the nature of the support schemes, while conventional plants are needed essentially as a residual to balance the system, in a quantity (and often of a type) determined by government decisions. In other words, markets are increasingly growing less effective in performing their essential functions – remunerating investment, providing for efficient operation, giving useful signals for consumers, optimising the plant mix – and are not sustainable without support. Proposed reforms that focus on just one of the challenges – for instance capacity markets to remunerate fixed costs – do not deal with the other problems and entail the risk of introducing further distortions.

The two market solution addresses these issues by creating separate markets for different sorts of power (‘on demand’ and ‘as available’) at both producer and consumer ends. For producers, dispatchable plants would operate in the ‘on demand’ or flexible market, be dispatched according to merit order when needed, and paid on broadly the same basis as at present. Intermittent plants would participate in the ‘as available’ market; in principle, they would operate as available and, at least initially, be paid a price reflecting the levelised cost of electricity from the particular source in question (with the price normally set via auctions at the investment stage). This is not in itself very different from the current Feed-In Tariff (FiT) auction arrangements which are used in a number of EU countries; however, the idea is that the differing costs and operation of ‘as available’ and ‘on demand’ sources would also be reflected in the retail market. Consumers would be able to select ‘on demand’ or ‘as available’ power (for which they would normally have separate meter readings) or combinations of the two sources. Initially – as at present – it is likely that price support (or public financing of some renewable costs) would be needed either at producer or consumer level to make the ‘as available’ offer attractive to consumers, but over time, as carbon prices increase and renewable costs fall, the support could be removed, creating a potential exit strategy (from subsidies).

The design is shown schematically in Chart 1.

Chart 1: Schematic of two-market design

As Part 2 explains in more detail, this design would:

• Provide signals for investment in both markets and, in the long run, enable investments in renewables and conventional plants to be remunerated solely from the market.

• Provide efficient signals for operation in the ‘on demand’ market and encourage consumers to maximise their use of ‘as available’ power (consumers would now have an understandable and effective choice, along with price incentives, to use this power; markets for demand response, on-site storage, distributed generation and the supply chain that supports these and other services, would develop in response to their preferences).

• Provide meaningful signals for consumers – in effect security would be privatised and consumers would be able to decide for themselves how far they were prepared to pay for secure supplies (system stability is a slightly different matter and would still be subject to system operator control). It would be possible for consumers effectively to use their own Value of Lost Load (VOLL) assessments in deciding whether to access the ‘on demand’ market.

• Provide scope for incorporating distributed resources, and network and transmission costs, using the same general principles.

• By these means, provide for overall system optimisation on the basis of consumer preferences.

• Provide an exit strategy – the support shown in the schematic above could be removed over time; intermittent generators would be able to remunerate their investments from the market once it had developed sufficiently to be understood and used by consumers.

• Deal with the problem of ‘pecuniary externalities’ by keeping the markets separate, at least in commercial terms, thereby providing investment signals and the prospect of fixed price recovery for flexible generation.

Part 2 elaborates this analysis in relation to the criteria set in the call for contributions.

Our proposals are based on a qualitative approach, rather than modelling – in the view of the authors it is not possible to undertake meaningful modelling at this stage, since we do not have adequate information about consumer preferences or likely technological developments. We are not therefore in a position to assess how to optimise the overall welfare impacts.

In essence, the design is a large scale effort to reveal consumer preferences and support the development of a supply chain that would respond to those preferences. The authors believe there is insufficient evidence available from current market structures on revealed consumer preferences and that the alternative of relying on stated consumer preferences that are incorporated into existing markets, eg via capacity mechanisms and centrally determined VOLLs, is liable only to introduce and rigidify distortions.

Fundamental changes are under way in electricity – the aim should be to let consumers drive the process rather than central decision-makers. The aim of the two market design is to open the way for them to do so.

2.1 Design principles

In Part 1, the conceptual basis and rationale for the two market design were described. This Part explores the modalities – possible options for implementing the proposal. It should be stressed at the outset that these are only tentative thoughts and by no means definitive. In practice, implementation could take many forms depending on systems of regulation, consumer preferences, the state of penetration and nature of renewable power within the power system etc. In addition, it would be natural for any fundamental change such as is proposed here to be preceded by pilot projects and practical experimentation to inform both the detailed design and the process of implementation. Many problems may arise in practice which have not been examined here. This Part is therefore aimed more at discussing some of the issues involved than at prescribing particular ways forward.

A two market approach could in practice take many forms. However, we would suggest that four underlying principles are essential to the concept as presented here:

At wholesale level:

• Economic separation of markets so that intermittent low and zero marginal cost plants which receive support from non-market sources do not create ‘pecuniary externalities’ for flexible plants with significant marginal costs.

• Price signals in both markets which are capable of remunerating investment and guiding operation.

At retail level:

• Separation of the consumer offer as between ‘as available’ and ‘on demand’ options.

• Cost pass-through from the two wholesale markets via the two separate offers respectively.

Provided these broad principles are followed, the market design could take many possible forms, some of which are discussed below.

2.2 Role of government and regulators

In the long run the aim of the proposal is for markets to be consumer-driven. The process could in practice start either from the bottom up, or from the top down. There are already, for instance, a number of experiments in place with power supplies of varying degrees of reliability or controllability – for instance the ‘Power-Off and Save’ scheme for residential customers in Ireland http://www.eirgridgroup.com/how-the-grid-works/power-off-save/ or the ‘demand turn-up’ scheme offered by the National Grid in the UK http://www2.nationalgrid.com/UK/Services/Balancing-services/Reserve-services/Demand-Turn-Up/ Such schemes are a valuable first step in accustoming consumers to the idea that reliability has a price while demand flexibility has a value. This is key to the overall proposal. Most consumers have hitherto been used to treating electricity as a homogeneous commodity, available at all times at the same price. Even when moves take place towards time of use pricing, they are insufficient to give clear signals because of the market distortions referred to above – and they do not convey the essential message that flexibility has both a cost and a value.

So the ‘bottom up’ schemes are a useful starting point. However, they are essentially ‘add-ons’ superimposed on a market which is failing to provide the right signals. A fully developed two market approach could not be established without government enabling measures in at least the following areas:

• Wholesale market design. Governments (and the European Union) are inevitably involved in establishing wholesale market design because of the special characteristics of electricity. Their agreement would therefore be needed for a full market separation to take place, as proposed here.

• Regulatory facilitation. National regulators would also need to facilitate the changes needed at retail level. The requirements would vary from country to country – for instance, many countries still retain some sort of retail price regulation and this would need to be adapted to the new market structures. Other countries set special metering requirements; but at the moment smart metering does not necessarily have the functionality to support a separate market of the sort proposed. Others forbid the bundling together of equipment and power supply. More flexibility might be needed in this area, depending on experience with pilot projects, to enable a supply chain to develop which would empower consumers to take advantage of the separate price offers (e.g. via home storage or interruptible equipment). While it is not clear at this stage what the right balance is likely to be as between centralised and decentralised generation and storage, it will be important to remove barriers to both sets of resource so that the necessary technological and commercial approaches can develop without distortions.

• Possible reform of support schemes. It is not an intrinsic part of the proposed reforms, but they would create an opportunity to refocus schemes of support for low carbon electricity. At the moment nearly all such support comes in two ways – support for low carbon generation via FiTs and the like; and support for energy efficiency programmes (Ecofys, 2014). The shift to a two market approach would enable some of this support to be shifted to the consumer end – to encourage demand for low carbon sources, the ability to use such sources, and the development of decentralised energy options, rather than simply subsidising the supply of centralised sources. This would give substance to the rhetoric of ‘consumer empowerment’; involve consumers actively in the process, instead of treating them as passive recipients of decarbonisation programmes; provide a means of gauging their true preferences; and help expand the range of options for system optimisation and management.

2.3 Wholesale markets and role of generators

As indicated above, the fundamental design principle here is separation of two markets – one for dispatchable plant with significant marginal costs and one for intermittent low and zero marginal cost plants. Of course, the distinction is not absolute – some low carbon plants are dispatchable; some have significant marginal or opportunity costs (eg biomass and hydro). Indeed, in a system dominated by hydro, the problems discussed in this paper might not arise in the first place and no market reform would be needed (though the scope for expansion of hydro capacity in Europe is thought to be very limited). On the other hand, it is at least possible that in the longer term all plants would have low or zero marginal cost, depending on the way technology develops – see ‘Long term evolution’ section below.

Other types of plant – eg nuclear – may fall somewhat in the middle, having relatively low marginal costs, high availability and a degree of flexibility. Rather than try to categorise all such variants centrally, the simplest option would be to allow plants to operate in whichever market they wished, with the proviso that the ‘as available’ market would only be available to low carbon plants (with government deciding the precise definition – for whether fossil plants fitted with carbon capture and storage would qualify as ‘low carbon’ while the ‘on demand’ market would only be for plants which could operate as and when required. (These would normally be dispatchable plants but, as discussed below, the criterion for access to this market could simply be the ability to supply on demand, so generators could use a combination of plants, including storage, for this purpose if they chose). Plants potentially capable of meeting both criteria could then choose which market to operate in. However, the decision would have to be consistent with the requirements of each market – see 2.4 below.

‘As available’ market

Pricing in this market would in principle be based on the current and future levelised costs of the source concerned, so that investment and operating costs could be recovered. In practice, it may not be possible to move to a self-sustaining market on this basis in the short and medium term, given that a large number of existing plants operate with FiTs or other forms of support. It might be necessary to provide two sources of income for such plants, as in the schematic above, designed so that they received the same agreed price as under their current arrangements, from a combination of the two income sources. It might also be necessary, for a time, to continue to provide support for new plants, for instance using auctions on the lines of the FiT auctions in some EU countries today.

The balance between the two sources of revenue (‘as available’ prices and renewables support) would be a matter of judgement. One possibility would be to change the basis of tendering for new renewable sources. The government could set a reference price for the ‘as available’ market, based on its assessment of the trend in long run marginal cost (or levelised costs) of intermittent sources. It would then be possible for generators to compete at auction on the basis of the premium needed over this ‘as available’ price.

If the government wished to take the route suggested above of shifting support to consumer level, this reference price could then form the basis of the offtake price for suppliers drawing power from the ‘as available’ market. The cost of the premium arising from the auction process could then be socialised across all electricity customers (or alternatively spread across the ‘on demand’ part of the electricity market, the entire energy market or recovered from general taxation). The idea would be to phase out the need for a premium over the longer run as the cost of renewable power came down; carbon prices went up; and consumers’ capacity to accommodate ‘as available’ power increased. Furthermore, over time a number of intermittent generators will start to lose the support they receive under time-limited FiTs or other schemes. Increasingly therefore, ‘as available’ generators would receive only one source of income – the price in the ‘as available’ market itself, which would in turn reflect consumer demand for that sort of electricity.

Dispatch in this market would in principle be largely automatic, at least initially – ie the plants would generally be dispatched on a priority basis (as is in principle the case across Europe at present) rather than by price, although over time price signals could be used to an increasing extent. In the short to medium term, while support schemes for plants of this type offer effective revenue guarantees, it might be necessary to offer constraint or other payments for plants constrained off the system due to local congestion or an excess of supply over demand. However, it is suggested that in the long run payments in respect of the supply/demand balance could be removed and replaced by the arrangements described below; potential generators would then have to assess the likely demand for ‘as available’ power, and the expected price in the ‘as available’ market, in making their investment decisions. There would be incentives for keeping such ‘as available’ power in balance with ‘as available’ demand as described in 2.4. Over time, therefore, it should be possible to remove any element of subsidy while ensuring that demand covers the cost of supply in this market – in other words this approach offers a potential exit strategy.

‘On demand’ (flexibility) market

The general concept is that this market would be, in a sense, a residual, providing the power needed to balance supply and demand across the system. Over time, as explained further below, it could be expected to shrink in size because of the increasing ability of consumers to manage their demand in such a way as to minimise their need to access this market. While the term ‘on demand’ is used, to make the parallel with the equivalent retail market, this is in effect a flexibility market.

Pricing and dispatch in this market would in principle be on the basis of short run marginal cost – in other words, this market would operate broadly as at present, except that prices would no longer suffer from the distortions caused by the presence of inflexible zero marginal cost plants. In principle, capacity payments would not be desirable or necessary – the market should give accurate price signals; and since consumers could now make their own decisions about reliability, there should be no long term need for governments or central authorities to make such decisions on their behalf. However, it is possible that the risks (of ‘missing money’ and a reluctance to invest) which have led many countries to resort to capacity payments would still be present, at least initially, as consumers and generators start to adapt to the new situation and it is likely that some governments would therefore still want to retain the option of capacity payments of some sort until the new system had proved itself. In the longer term, the fact that consumers would have both greater responsibility and a greater range of options for managing their own security should make it easier for governments to allow scarcity pricing in the ‘on demand’ market – consumers would only be exposed to this market to the extent that they chose to do so – which should in turn reduce the perceived risk of government intervention or ‘missing money’.

Access to this market would in principle be for reliable plants only. The aim would be to ensure that both the capital and the marginal costs of reliability were covered and that the srmc based pricing system would give incentives to minimise costs. As discussed above, the authors suggest providing a choice of market for generators so there would not need to be any specific technology criterion for access; instead, reliability could be incentivised in the same way as happens at present with some capacity and reliability markets – that is, the system operator could impose a requirement for access to this market that, for a given number of critical periods (over, say, each year) generators would need to be able to guarantee their availability in response to a few hours’ notice. If they could not in the event deliver at those times, they would face penalty payments. In order to prevent gaming, it would follow that the choice of market could not be made in the short term – otherwise generators could nominate themselves when they felt confident of their short term availability, even if they could not guarantee that availability in advance (and so were not providing truly firm power).

There remains a question of the price available to intermittent plants which were in effect ‘spilling’ excess generation (and whose capital costs were covered from the ’as available’ market). The general principles set out here would imply that such plants would not have direct access to the ‘on demand’ market and that there should be incentives, as set out in the following section, to discourage them from such ‘spill’. On this reasoning, they should not therefore be direct participants in the ‘on demand’ market and should not receive the (normally high) prices in this market.

An alternative view is that there is an opportunity cost in not taking advantage of the availability of low marginal cost power at any particular time; a possible approach would therefore be to allow excess generation from the ‘as available’ market to compete in the ‘on demand’ market on the same basis as other generation. However, the central argument of this paper is that short run marginal costs do not represent the true costs of such intermittent sources and that allowing direct access to the ‘on demand’ market would create pecuniary externalities for generators which are seeking to recover both capital and marginal costs from the ‘on demand’ market. In Section 2.4, we discuss this further and propose ways of discouraging ‘spill’ while not losing the operational benefits which arise when renewables displace fossil-fired generation.

2.4 Balancing supply and demand and role of the system operator

One of the objectives of the two market approach is to distinguish between overall security (the balance between supply and demand) and system stability (the specific system risks arising with electricity because of the difficulty of storage, the very short timescales involved, and the network nature of supply). The latter risks are essentially ‘public goods’ in the sense that they are non-excludable; they therefore need to be supplied at system level rather than individual level, and to be subject to a degree of central control and management – though of course the sources themselves could be decentralised. However, it is the contention of this paper that the former risk, that is the reliability of individual supply, is essentially a private matter – with the growth of information technology and controls, it is quite possible for one consumer to have their supply interrupted or curtailed while their neighbour does not.

The two issues are therefore dealt with separately below.

Balancing supply and demand

In very broad terms, balancing supply and demand would take place as at present. Different markets operate in somewhat different ways and, as at present, two broad options would be available:

• Pool: In this model, the ‘on demand’ market could be centrally managed and run by the system operator on a pool basis, using merit order dispatch to ensure that the lowest cost generators at any particular time were the ones called on to generate. Contracts for differences between generators and suppliers, or similar mechanisms, could then be used by suppliers (retailers) to manage their own demand and provide price security. On this model the ‘on demand’ market would be a pure residual market only – it would be designed only to meet the balance of demand above that accounted for by ‘as available’ generation. Meanwhile, ‘as available’ generators would be free to feed power into the system as they wished, except where the volume had to be constrained because of an insufficient level of overall demand. In other words, on this basis, balancing would be at system level rather than individual supplier level. This approach would not necessarily ensure that ‘as available’ generation would be exactly in balance with ‘as available’ demand but it would still allow appropriate price signals to be given, for instance, via contracts for differences (as in the days of the Pool in the UK), to encourage a broad balance at the wholesale supplier level between retail suppliers and generators. The mechanisms for doing so might follow the general lines of the model set out below for self-balancing, though the approach could be broader brush (eg through use of ‘profiling’ and averaging across demand categories) while still giving reasonable cost signals. This approach might be simpler to introduce; because of the central management of the system operator it might also reduce concerns about security. However, in the longer run, given the objective set out above of ensuring direct pass through of cost and price information between the separate upstream and downstream markets, an approach based on the ‘self-balancing’ model set out below might be more effective and give more accurate price signals.

• Self-balancing: On this model, the two markets could each be run on the basis of ‘self-balancing’ under which individual generators and suppliers would need to keep their contract positions in balance within each market (ie they should aim to ensure that the same volumes of power were supplied to, and offtaken from, each market respectively). Generators could self-dispatch; generators and suppliers could trade power up until gate closure but then be exposed to penalties, or the risks of the balancing market, for any imbalance in their contractual positions. Again, this would be very much as happens at present in markets like that of the UK, with the major difference that there would now be two separate markets to consider. To be clear: on this model, both markets would need to be in balance at the level of market participants – ie generators and suppliers would need to match their generation and offtake in each market. Some possible ways of dealing with this are discussed below.

When the markets were run on a self-balancing basis, there would need to be arrangements to give suppliers an incentive to keep demand from their own ‘as available’ customers in balance with their contracted offtake from that wholesale market. The precise means of doing so would depend on the metering and information technology available. But if the proposal below is followed – that ‘as available’ customers would have separate metering for this sort of supply (which, depending on the functionality of existing meters, might require new meters to be installed or new software to be programmed) – it would in principle be possible at any one time (or during any particular charging period) to determine how much ‘as available’ power was being offtaken by those customers. Similarly, ‘as available’ generation supplied to the system would normally be measured via real time metering, so it would be possible to reconcile the contractual position of any particular supplier and generator to determine whether they were in balance. Of course, suppliers and generators could trade up until ‘gate closure’ to try to balance their positions.

There would then need to be incentives for suppliers to remain in balance for this sort of power. One way of doing so would be to require them to pay ‘on demand’ market prices for any shortfall in their own contracted position – ie they would in effect be using the flexibility market to supply their ‘as available’ customers. Since the overall system design should ensure, or at least create the serious risk, that these flexibility market prices would be considerably higher than the ‘as available’ prices, there would be incentives not to have a shortfall of this nature.

The risk of an excess (that is of having contracted more ‘as available’ power than their customers were consuming) is not symmetrical – if this received the potentially higher price set by the ‘on demand’ market, there would be incentives for over-generation. The principles set out above would not justify paying such a price or giving access to the ‘on demand’ market for unreliable power. There would be various ways of creating an appropriate disincentive – for instance, if, in the introductory stages, the ‘premium over offtake price’ approach to auctions described above were to be adopted, excess power could simply lose this premium. Over time, the penalties would be expected to increase. In the long run, excess generation of ‘as available’ power over the equivalent sort of demand might need to be penalised by stronger balancing market signals – for instance, it could be remunerated at the level of the ‘as available’ generator’s short run marginal costs (usually very low). The aim would be to create incentives for generators and suppliers to keep in balance and to invest for the long run in such a way as to match demand in the appropriate market, but not to create incentives of such a penal nature that they prevented the use of excess ‘as available’ power in the flexibility market - if excess low carbon power were available, despite the incentives to keep in balance, it would be desirable for it to run ahead of the higher carbon sources in the ‘flexibility’ market.

For both generators and suppliers, the commercial side of balancing could probably not in practice take place in real time – there would need to be reconciliation of contract positions after the event to establish where they were short or long of the relevant type of power.

System stability

As indicated above, the proposal aims, among other things, to distinguish between security (in the sense of having enough supply to meet demand) as a private good and system stability as a public good. The aim would be essentially to ‘privatise’ the former set of risks while leaving the latter risks to be managed centrally by the system operator. While distinguishing between security and system stability risks is not an absolute science, the problem is not new – for instance, in the UK, in addition to the normal wholesale and balancing markets in which plants of various sorts compete, the National Grid operates a special Short Term Operating Reserve to cover short term imbalances and offers contracts for frequency support, black start and other specialised services. The issue is essentially one of timescales. As IT advances continue it can be expected that ‘gate closure’ will take place nearer and nearer to real time; the ‘private’ part of the market will therefore continue to expand and the ‘public’ part contract, though there will no doubt there will always continue to be a residual need for special arrangements to ensure system stability in the very short term. Reliability in this narrow sense will probably continue to be supplied (or at least managed) in broadly the same way as at present – though no doubt with an increasing tendency to balance at a more local level and to include distributed energy resources. It is suggested that the costs should largely be spread across the ‘on demand’ market - although, as discussed above, it would be necessary to incorporate incentives for generators of intermittent power to control their output in such a way as to minimise problems in the ‘on demand’ market and there would need to be arrangements to reflect local congestion, as discussed below.

Introduction: overall approach

The proposal is novel and the thinking is spelt out in some detail here, with the same proviso as above – that it is the underlying aims that matter, not the specifics of implementation. The basic aim of this model is to present consumers with a simple choice between different sorts of supply – ‘as available’ power at a low and relatively stable price and ‘on demand’ power at a significantly higher and more volatile price. As with wholesale markets, there would be many ways of introducing such a choice.

One option would be to provide for different classes of electricity supply, through separate meters, at different prices. There is a rough precedent in the UK ‘Economy 7 tariff’ and its predecessors, which offered a lower night-time rate, via separate metering, mainly intended to encourage electric storage heating. Prices under this tariff were significantly lower (at about 40% of daytime electricity). As Chart 2 indicates, this price differential produced very significant changes in demand levels and patterns (peak demand for consumers on the Economy 7 tariff – LP2 – is over twice that of the average consumer on the daytime tariff – LP2 – and the peak is shortly after midnight, rather than at breakfast time). These impacts are orders of magnitude greater than what has been achieved with traditional time of use pricing; impacts on this scale would be needed if demand is going to make a significant contribution to system balancing in the future.

Chart 2: Demand profiles of different consumer classes in the UK

With the development of smart metering, it is conceivable that separate meters and circuits would not be needed. Instead, consumers could have the option of using appliances fitted with micro-chips which could react to the presence of ‘as available’ supply and be designed to make best use of it. However, the success of the Economy 7 approach suggests that flexibility may be needed on the issue of ‘bundling’ together equipment and tariffs in order to enable such a capacity to develop. Since it would not be possible to take full advantage of the ‘as available’ market without having the necessary controls, storage, etc, there could be a barrier to the development of this market unless it were possible for suppliers to offer a full service for this purpose, including appropriate equipment, and for aggregators to manage supplies for groups of consumers in such a way as to optimise the potential flexibility.

There is no intention that consumers should be restricted to one sort of supply only; as with Economy 7 power, consumers could have access to both sorts of power if they had the right sort of equipment or metering; such consumers would receive the ‘as available’ price so long as sufficient power of that category was being generated. When demand rose to a level where both sorts of power were being produced, ‘as available’ consumers could draw on the ‘on demand’ market as necessary (ie to the extent that their supplier did not have sufficient ‘as available’ supply to meet demand). Smart metering would need to be capable of signalling the availability of the different sources; consumers might want the choice, for nominated equipment, of having automatic cut-off at high price periods, or simply not to use any of the ‘on demand’ power. Consumers who did not have the right sort of metering and equipment for ‘as available’ power would pay the more expensive ‘on demand’ price, incorporating wider system costs of reliability and flexibility, along with what is expected to be a rising carbon price. There would therefore be a big incentive to have an ‘as available’ supply (and if governments chose to do so, it would be possible for them to help with the extra costs of providing the equipment needed, for low income consumers in particular).

As available offer

Pricing: it is envisaged that pricing in this market would reflect the offtake price from the ‘as available’ wholesale market (as described above), with a margin for supplier costs. As discussed above, carbon costs, along with those network, balancing and other costs associated with flexibility and maximum demand would be borne primarily by the on demand market. Final electricity prices should therefore be relatively low in the ‘as available’ market and the differential would be expected to increase further in its favour over time. It should therefore be an attractive option in itself.

As suggested above, if governments go down the route of refocusing low carbon support, they could make the option additionally attractive so as to encourage demand for such supply. This could involve support for the price to suppliers of ‘as available’ power, as described above, which could then be reflected in lower prices to consumers. The cost of the support could then be spread across the ‘on demand’ market (as in effect happens at present), across all energy consumers, or incorporated in general taxation and spread across society as a whole.

Availability of this sort of power could be calculated in one of the two ways suggested above: either it could reflect the volume of power being traded through the ‘as available’ market in general and then be purchased by or allocated between suppliers (as might be the case if, for instance, the government acted as an intermediary by purchasing the ‘as available’ power itself – in effect, in many regimes, it is the government or an agency which acts as counter-party to the relevant contract); alternatively, it could be directly related to the contract position of the individual supplier (ie the amount of power they could sell at an ‘as available’ price would correspond to the amount of such power they, or generators with which they had contracts, were producing at any one time). This means that they would have an incentive to draw from a diverse a range of sources to increase the overall reliability of their supply. Consumers could then choose between different suppliers with different trade-offs between reliability and price.

On demand offer

Pricing and reliability: These could very much follow the same patterns as at present. Prices would reflect the costs of supply, including carbon prices, in the ‘on demand’ market, along with most balancing, congestion and network costs, apart from those specifically allocated to the ‘as available’ market’ – see below. Reliability, at any rate initially, would also be as at present – if governments wished, they could set standards centrally and the market would be designed to deliver that degree of reliability. Prices in the wholesale market could be expected to vary significantly with the level of demand – with the removal of the ‘pecuniary externalities’ associated with the present mixed market, prices would be at a level which would in principle cover the short and long term costs of all generators, absent excess capacity – so time-of-use pricing could also be usefully adopted in retail markets on this approach and would give genuine cost signals.

It would then be possible to package up the different offers in various ways.  For instance, some consumers might want to be able to access the ‘on demand’ market from time to time but not to face extreme price volatility.  One could then envisage packages which offered, in effect, options on firm power – say a combination of ‘as available’ power plus an option of up to 1,000 hours a year of ‘on demand’ power at an agreed price.  This sounds complicated but it is not very different from the structure of many telephone contracts under which the customer buys a basic number of gigabytes and pays a significantly higher price for any excess over that limit.  However, this is only an example of the sort of arrangement which might be considered: the authors are not aiming to prescribe particular models – only to create the circumstances where there will be real incentives for the development of innovative approaches to pricing at retail level.

Development of supply chain and evolution of consumer behaviour

One of the aims of this proposal is to provide incentives for the development of a supply chain to support a new approach to consumer behaviour in a low carbon world. Manufacturers of appliances, storage capability, in-house displays and meters would have a target market, corresponding to demand for ‘as available’ energy. There would be a clear offer for consumers – the capacity to reduce or eliminate their exposure to the expensive ‘on demand’ market. Retailers, aggregators and other intermediaries would develop their service offerings to reflect customer preferences in terms of their willingness to pay for different kinds of energy service.

Another key goal is to influence consumer psychology and develop consumer understanding of the nature of electricity supply. At the moment, there is a clear tendency, among both policymakers and consumers to think that ‘electricity is electricity is electricity’. In fact, of course, the value (and price) of electricity is dependent on many factors such as location, time and the overall state of the system (Hirth 2015, pp 925-939). This can lead to distortions of perception; for instance, many policymakers believe that once new renewable sources reach ‘grid parity’ (where the levelised cost of generation is comparable with that from conventional sources) they will be competitive and there will be no further need for government support. Increasingly, however, as the levels of penetration of intermittent renewables grow, system integration costs also assume greater importance, and levelised costs cease to provide a meaningful reference point. A pricing mechanism which clearly showed the difference in value between reliable and intermittent forms of supply might help promote a wider understanding of the value of electricity and the nature of electricity as a system. Of course, this has to be done in a way that is easy for consumers to understand and which gives consumers an easy way to participate in the ‘as available’ market.

This change in perception would make it easier to introduce an undistorted pricing system, under which the value of different sources was determined on a level playing field, across the whole electricity system, from embedded generation to storage, network expansion and so on. This would in turn enable markets to perform the task of system optimisation – of determining the right balance between sources such as storage, demand response, interconnections and other options. At present all these sources are regulated differently in the pursuit of specific policy objectives; there is no real attempt to ensure overall system optimisation.

Demand response, storage etc

Another objective (which may seem perverse at first sight) is to start to move away from concepts like ‘demand response’. In many ways this is a confusing term for consumers. In most markets, there is no such concept – despite the fact that, in a sense, demand response is a feature of all markets, via the familiar forces of supply and demand. In normal markets however, demand response is a consumer decision to consume or not, rather than something defined as a product to be purchased centrally. From this perspective, the fact that ‘demand response’ is seen within electricity as a product is a symptom of the over-centralised approach to markets; it may have been a necessary concept at a time when information technology was not sufficiently advanced to enable the ‘privatisation’ of energy security, but those days are arguably now over. From this perspective, the focus should not be so much on ‘demand response’ as a product as on reforming electricity markets, for instance in the ways proposed in this paper, so that they give proper signals about the cost and value of flexibility, along with incentives for consumers to determine the level of reliability they actually require rather than whatever has been set by some central authority. Increasingly it should be possible to move towards a situation where the requirements of system stability are dealt with separately as a residual technical function of the system operator; and ‘demand response’ as a product becomes one of the ancillary services provided in this context.

In any event the meaning of the term ‘demand response’ is not clear cut; defining a specific product may well distort market development and narrow the range of options. Traditional definitions focus on demand shifting (ie away from times of system stress to other times) but in some countries, like the UK, a wider definition is now generally used. After all, if the aim is to reduce demand at times of system stress, why does it matter whether that demand is shifted to some other time or forgone entirely? So demand reduction is sometimes included in the envelope (taking us on to territory more frequently associated with energy efficiency). Furthermore, the focus on shifting demand from peak times may no longer be the right one; in a system dominated by intermittent sources of generation the requirement may well relate to times of low supply as much as to times of high demand, and more generally to the need for flexibility at any time.

In practice, the definition is often even wider – for instance, the UK government has defined demand response (Frontier-DECC (2015) as ‘all actions that reduce demand from the transmission system at a particular moment in time’, a definition which includes dispatchable distributed energy resources and storage. In short, ‘demand response’ can have a variety of definitions – the debate in different contexts may be focusing on different things and it may even be questioned whether the concept of ‘demand response’ is a useful one. Some countries, like the UK, have tended increasingly to use the wider (and even vaguer) term ‘smart power’, as discussed below. The very process of trying to define a particular product called ‘demand response’ against this background is inherently liable to distort the market.

The two market approach has as one of its objectives to simplify this confusing, over-centralised and over-technical picture and bring electricity markets closer to other markets, where demand response is a description of consumer behaviour, not a defined product. Under the approach, consumers could decide what level of reliability they wanted, and pay accordingly. It is something to which they are already accustomed in many other areas where supply is fixed in the short term, while demand varies. For instance, for air or rail travel, hotel costs etc, it is common for prices to reflect the level of demand and congestion at particular times and for customers who want flexibility (eg in their time of travel) to pay more than customers who can accept what is available. Customers can alter their behaviour (eg travelling at off-peak times) in response to these price signals. This is in a sense a form of demand response – with the big difference that it has developed organically via the interplay of supply and demand, rather than being defined as a requirement by some central authority. It would still be possible for specific and defined demand-side products to be developed (eg in support of system stability, or as part of an aggregator’s portfolio) but the overall aim would be to focus the notion of demand-side flexibility and reliability on the two markets rather than on specific products.

Network costs

The same general principles that underlie the design of electricity markets themselves could be applied to network costs – that is, the costs could be shifted towards maximum demand (via use of system charges) rather than throughput. The implications would vary depending on the starting point in particular countries in terms of network pricing and the availability of maximum demand metering. The general principle would be that costs should fall on the market (and consumers) which gave rise to those costs. In line with the starting point set out above, it is likely that most network costs would be allocated to the ‘on demand’ market, which is what drives maximum demand on the system – the main cost driver. If metering permits, the costs could then be allocated according to an individual customer’s maximum demand at the time of system peak demand, but a first step could be to ‘socialise’ most use of system costs across the ‘on demand’ market. Some costs would still fall on both markets of course – for instance, those costs which vary with throughput (eg losses) and those which are caused by local congestion, in cases where this is due to the ‘as available’ market. More generally, use of system or ancillary service charges should be designed to give consumers incentives to manage congestion.

Aggregators and Intermediaries

Although – for the sake of simplicity – this paper refers generally to ‘consumers’ it may well in practice be aggregators, suppliers or other intermediaries who are taking advantage of the opportunities and challenges thrown up by the new market structure. For instance, it may be cheaper for a supplier to offer reliable power to consumers by buying power themselves in the ‘as available’ market, then using storage to match supply and demand and limit exposure to the ‘on demand’ market, rather than for a consumer to invest in storage themselves. Alternatively, it may simply be easier, because of the behavioural issues outlined above, for the consumer effectively to contract out the task of having to think about these problems to a supplier or aggregator. There could also, of course, be various intermediate solutions, such as neighbourhood or community schemes or those involving large blocks of apartments or other local systems.

International trade and the EU

As with other aspects of the proposal, various options would present themselves in this area. In principle, it would be possible, certainly in the long run, to enable trade to take place separately in both markets as long as the rules for access to the markets were consistent across jurisdictions. However, the fact that the ‘as available’ market is affected by the systems in place for supporting the associated generation might, in the near term, make individual countries reluctant to open up to free trade in this area. The problem itself is not new – at least to date, support schemes for renewables have been limited to suppliers within a member state, despite the distortions of trade this entails, and these distortions have been one of the factors undermining the overall EU market. Over time, if and as access to national support schemes opens up to nationals of other member states, this problem should decline. (In a way, the problem is analogous to that of capacity payments – different national schemes have the effect of undermining the EU market. But it is argued above that it should be easier to do away with capacity mechanisms and renewables support under a two market approach.)

Furthermore, it is important to stress that these are not problems created by the two market approach; they are created by the different national schemes for renewables or capacity support. One result of a move to a two-market approach may in fact be to make more transparent the distortions which arise from the present systems of support. That transparency may create political barriers to trade in the short term (in which case, EU trade might need to take place on the basis of trade between ‘on demand’ markets only). But that in turn should help encourage the move proposed by the Commission towards the harmonisation of national systems of support, in the interests of a freely operating Single European Market.

2.7 Transition and the long run

The two market approach is designed to a large extent as a transitional measure in the sense of being a process of discovery –it will take time to set up the systems, hardware and consumer understanding for a fully self-sustaining low-carbon power supply. In particular, it will take time to delineate the demand side resource potential and it will require systems to be in place that make it simple and practical for consumers to engage with this new area. It is therefore difficult to be definitive about the long run.

Uncertainty also applies to technology on the supply side: it is in theory possible that in the long run most plants will be primarily fixed cost. In the view of the authors, that outcome is unlikely. It is almost certainly always going to be cheaper, for straightforward economic reasons, to provide flexibility via plants that have a relatively high marginal/fixed cost ratio, like fossil or biomass combustion plants – or have a significant opportunity cost because of the storability of their power source, like hydro – rather than via plants whose costs are almost entirely fixed. However, should the situation arise where all plants were essentially fixed cost, it would in principle still be possible to apply the overall two market approach, with prices in the flexibility market reflecting scarcity or congestion. Clearly prices could in theory rise to very high levels on this scenario, but the aim is by that time to have created a capacity for self-supply or demand management amongst consumers which would at least mitigate the consequences (or, in the view of the authors, mean that the scenario itself was very unlikely – ie short term demand response is likely to be more economic than short term use of generation sources whose costs are entirely fixed).

With these caveats, it would be the authors’ expectation that in the long run, government intervention in the electricity market could be reduced to setting the overall framework conditions. Policy intervention would continue to be needed in order to incorporate the carbon externality and ensure that carbon targets were met. However, this could be done either by a carbon price or (the authors’ preference) through tradable carbon intensity targets (Buchan and Keay, 2016); the use of one of these options would get away from the need to support particular technologies or sources (like storage or demand response) and allow markets to select the lowest cost options. In other respects ,the market should be self-sustaining. Over time, the consumer trade-off between security and price should be well-established, and market prices should be capable of signalling the need for investment in different power sources, including consumer-side sources such as in-house storage and consumer demand management.

2.8 Conclusions

Electricity markets are broken; they no longer fulfil their primary functions of providing appropriate signals for producers and consumers. The problem arises from a combination of changes in technology (from predominantly marginal cost plants to predominantly capital cost plants) and of policy (support for intermittent renewable plants) which undermine traditional market structures. In the view of the authors, markets will require fundamental reform to resolve the problem. Existing market structures are inevitably leading to greater central intervention – support for renewables and the creation of capacity markets. There needs to be a shift in emphasis which will enable consumer preferences to be expressed clearly and drive overall market development. The reforms needed will require not just a change in market design but also in consumer attitudes to electricity – this will necessitate a relatively simple and comprehensible basic offer at consumer level.

Against this background the authors propose a new approach to market design which will enable intermittent renewable sources to be accommodated; maintain overall system reliability while enabling consumers to put a value on their own supply security; provide clear signals to generators for investment and operation; and provide an ‘exit strategy’ allowing government intervention to be limited in the long run to the setting of framework conditions only. In the view of the authors, no other proposal put forward to date can meet all these objectives.

BIBLIOGRAPHICAL REFERENCES

Buchan and Keay (2016) Europe’s Long Energy Journey: towards an energy union? Annex 2 Buchan D. and Keay M. Oxford University Press 2016

Ecofys (2014) Subsidies and Costs of EU Energy Report prepared for the European Commission, Ecofys, October 2014

Frontier-DECC (2015) https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/467024/rpt-frontier-DECC_DSR_phase_2_report-rev3-PDF-021015.pdf

Hirth (2015) Integration costs revisited – An economic framework for wind and social variability Renewable Energy vol 74 pp 925-939, 2015

OECD/NEA (2012), Nuclear Energy and Renewables, Paris 2012, p 34-37.