Effective Algo Trading Surveillance for Short Term Power Markets (Part 1)

Algorithmic trading (‘Algo trading’) in European physical power markets continues to grow rapidly. Yet so far Algo trading in physical products has maintained a relatively light regulatory footprint in contrast to similar activity in financial products which is tightly governed by RTS 6 [1] of MiFID 2. With REMIT 2 [2] approaching, this is set to change with new compliance rules around Algo trading in wholesale energy products being introduced. This will leave many firms needing to solve the dual challenge of monitoring highly complex short term power markets in parallel with the challenges brought by Algo trading. In this two part series we take a closer look at how these challenges interrelate. In Part 1 we explore the practical challenges compliance officers should consider as they extend their monitoring coverage into this area. In Part 2 we will examine governance considerations in this area.


Ask any Compliance manager responsible for European power market surveillance – they will concede that there are unique features and complexities inherent in short term power markets like EPEX Spot and Nord Pool which make effective surveillance challenging compared to standard futures products like those traded on EEX and ICE. Fundamental to this challenge lies the nuanced detection capability required to detect abnormal behaviour in the highly granular SIDC[3] market (formerly XBID) with unique market and product features such as the shared order book and quarter-hourly products and associated block contracts. Many firms active in these markets have not yet made substantial attempts to monitor them, and for good reason. Given, unlike under MAR, there has not previously been a regulatory obligation to monitor this activity combined with the complexity of doing so, most firms opted to exclude physical power from their surveillance programmes. This paradigm seems set to change. REMIT 2 will bring renewed focus on physical power causing many firms to reconsider their position in this respect, particularly those engaged in Algo trading. But what are the likely challenges that need to be overcome and what should Compliance managers be considering in the lead up to REMIT 2?

Data challenges for effective surveillance

Effective surveillance is highly dependent on the data used by the surveillance tool. For Algo trading this data must be highly accurate, complete and timely as well provide detailed information about the orders and trades themselves. At present it is debatable whether the current short term power exchange data models and APIs are fit for purpose, and entirely able to support effective surveillance per the examples outlined below.

In the first example, surveillance models should be able to distinguish between human trading and Algo trading, particularly when trading activity in a common market within a firm is split across the two. The ability to distinguish between the two from a data perspective presents several technical challenges. Unlike MiFID 2, REMIT does not mandate the reporting of the Execution Decision maker field where Algo IDs would otherwise be reported. Exchanges like EPEX Spot and Nord Pool do not currently accommodate this field within their data structures. This forces firms to devise workarounds that usually involve the use of free text fields (like Sub Party) and an associated front office process is needed to ensure that such fields are consistently populated. The ability to tell Algo transactions apart from human transactions is important for two reasons. First it allows the surveillance tool to be separately calibrated and/or analysed between each type of trading. Secondly, compliance and front office should be aware when two or more Algos or human traders are interfering with one another in the same order book.

The second example involves the “Aggressor” flag i.e. the indicator of whether your traders initiated the execution of a trade. Both the EPEX Spot and Nord Pool APIs do not currently make this available in the Shared Order Book phase of the SIDC market but rather only in the local phase i.e. typically the last hour of trading before gate closure. The aggressor status of an order provides important contextual information about the transaction to help the surveillance tool identify suspicious behaviour. Not having this information can be detrimental to the tool’s effectiveness, particularly when Algos are involved, and can make the surveillance analyst’s life a lot harder when this status indicator is not easily identifiable during alert analysis.

A third and final example relates to the exchange APIs and their ability to provide sufficient information about the various potential underlying transaction types being executed under a single Algo trading strategy. For example, some third party Algo trading platforms offer order execution strategies that might combine iceberg orders with limits orders. While the limit orders and icebergs might be individually identifiable from the data provided by the exchange API, there is usually no way to understand that these two order execution strategies are related to the same Algo strategy. Not having the ability to understand this connection impacts surveillance effectiveness and loses potentially critical contextual information for the surveillance analyst. This begs the question that if this information is not available via the exchange API feeds, where is it available and what are the possibilities for ingesting such data into the surveillance tool? Many Algo execution platforms (such as Volue and PowerBot) will hold this data by necessity but is it feasible to use it as either a primary or corroborating data source feed for surveillance? This issue is likely to get a lot more attention in the months and years ahead.

Given that many data deficiencies impacting surveillance are unlikely to be resolved by prescription under REMIT 2, exchange members should be placing pressure on the various physical power exchanges to do their part in helping to address them.

Timestamps, order sequence and latency

Timestamps for orders and trades take on a new importance with Algo trading. This is particularly the case for High Frequency Trading (HFT) which is becoming increasingly relevant for short term physical power markets. For market abuse pattern analysis to be accurate and meaningful, the sequence in which orders and trades are identified by the transaction surveillance solution is critical. This is important for establishing causality where, for example, a potential case of Layering and Spoofing requires a precise understanding as to when an order hits the order book, and in what sequence other market orders and trades occurred. This is fundamental to identifying (and then analysing) whether the Algo was responding to legitimate messages from the exchange, or if the behaviour was intentionally abusive. This is difficult to establish when accurate time stamps and order sequencing are not available and where milliseconds can make all the difference. While an exchange’s regulatory team may see what seems like abuse behaviour from their perspective, the firm under suspicion has only their internal timestamps and robust transaction surveillance analysis to rely on for defence.

Central to this issue is latency, a term used to describe the various delays from the point of initiating and order until its execution, and sequencing. Different degrees of latency exist at various points in the order message flow (e.g. order entry latency, transmission latency, exchange latency and round-trip latency). Latency includes the time taken between the Algo trading software, the exchange, the data adaptors etc. usually resulting in multiple timestamps for the same order or trade. Ensuring that correct timestamps drive the surveillance engine is critical for effective surveillance. A simplified example is illustrated in the diagram below.

An order is sent by the exchange engine (1) and received by the Algo platform (2). The Algo responds to the exchange order by generating its own order which it sends to the exchange (2). In the meantime the exchange engine sends a second order (3) unrelated to the first which crosses (4) the order sent by the Algo i.e. ‘Trade Latency Crossing’. The Algo’s order arrives at the exchange engine (5) at a similar time to the second unrelated order from the exchange (3) potentially making the two appear related. While there is no relationship between the Algo’s order (2) and the second order from the exchange (3), it appears that the Algo was responding to (3) when it was in fact responding to (1). Were the Algo’s order to appear suspicious, the firm may need to prove there is no relationship between the two – the accuracy of timestamps would be central to this defence.

How data is acquired for the transaction surveillance engine often drives whether latency is a problem or not. Presently there are two main ways firms may acquire the relevant data for surveillance. The first is from an end of day file provided by the trading platform or data vendor containing own orders and trades, as well as anonymised market orders and traders. The second is when a file is compiled continuously throughout the trading day from, for example, a live exchange feed. It is typically the second approach which introduces the latency and sequencing challenges, particularly if it involves combining data from multiple sources (although the disjoints in sequencing and timestamps might also impact the first approach as well depending on the data source). Perhaps encouragingly, short term power exchanges like EPEX Spot and Nord Pool provide a single live feed and API for both private and market data. This should go some way in reducing the risks posed by time stamps, but not entirely.

It could be erroneously concluded that this issue is relevant only for HFT. It is not. Firms engaged in non-HFT Algo trading in short term power markets should review their data sources to ensure that their surveillance tools are provided with the correct data to avoid the consequences of getting this wrong.

Performance and Detection Increased Data Volumes

Algo trading typically causes a significant increase in transaction activity, particularly in order volumes. As most surveillance tools use indicators such as order-to-trade ratios (OTRs) in their detection logic, Algo trading can lead to spikes in such alerts. While some surveillance tools are able to self-calibrate as they adjust to ‘normal’ conditions, many require regular manual adjustments to prevent false alerts. This is particularly so if Algo trading strategies are applied at irregular intervals, for instance at specific times of day or only on selected days of the month. The need for continual manual adjustments consumes resources and potentially weakens the effectiveness of the surveillance. In addition, many firms combine human and Algo trading in the same instruments and markets, often as part of different trading strategies, desks or entirely separate parts of the business. This can also exacerbate the calibration issues.

Macro trends in European power market are also a factor. While levelling out in 2023, the number of SIDC trades across Europe have grown exponentially since 2018 based on numbers recently published by ACER (see chart below).

These statistics relates to trades only but order volumes have grown equally as rapidly with ACER indicating that orders placed on organised trading venues represent a significantly high percentage of total reported records. Algo trading penetration in short term power markets is also expected to increase significantly in the coming years. In 2023, one of the major power exchanges hypothesized that virtually all short-term power trading would be conducted via automated trading methods within three years. Compliance teams must be equipped with the tools and capabilities to ingest, process and make sense of this rapidly growing data volumes to have any chance of staying ahead of their regulatory stakeholders.

Order Book Behaviour and Data Visualisation

Following on from the data volume topic above, Algo trading in highly granular short term physical power markets creates a data density quite unlike any other market. Beside the challenges posed by detecting suspicious behaviour within this complex hourly, half hourly or quarterly data, once detected the surveillance analyst will find many challenges in analysing an alert. The need for a granular data visualisation capability, including of the order book, to help them identify and interpret patterns is critical. Having the ability to step through the evolving order book in sufficiently small time-steps to capture the full context of an Algo’s transaction decision making is paramount. Order and trade placement at any given moment provides critical information about the Algo’s behaviour and whether it is engaged in untoward activity. It goes without saying that relying only on raw data tables or spreadsheets to perform such a task presents a serious impediment to effective surveillance analysis, if indeed providing any value at all.

Data historization, the Achilles Heel of Short Term Power Market Surveillance

While not unique to Algo trading, for transaction surveillance solutions to be effective they require transaction data history to be maintained at a sufficient granularity so that “normal” behaviour at this level can be benchmarked by the solution e.g. how does DE H22-23 hour typically behave? This “historization” of data allows a surveillance tool to compare and identify abnormalities across parameters such as trade and order volumes, counts, and prices. This concept is not straightforward however when it comes to short term power where intraday power products are ephemeral, or temporary in nature, making them hard to track. The challenge arises when, for example, a quarter-hourly intraday product is assigned a trade ID which is then subsequently replaced with a new trade ID for the next equivalent delivery period. Without product history there simply cannot be an accurate transaction surveillance calculation. Effective surveillance in these markets depends heavily on this relatively benign concept however most surveillance solutions are simply not able to handle this. As in so many cases, despite this topic not being unique to Algo trading, the issue is exacerbated when Algo’s are applied to short term power trading.


Compliance officers responsible for the oversight of European power trading face the daunting dual challenge of monitoring physical short term power activity, a new proposition to most, alongside increasing levels of Algo trading in these markets. While many energy trading organisations have introduced surveillance programmes, these are often limited in scope to covering products and markets falling under MAR. REMIT 2 will force the hand of many firms active in physical power who are likely to encounter the many challenges outlined in this paper. While not all of these challenges can be solved immediately, it is clear that specialised tools are needed. Firms impacted by REMIT 2 are strongly advised to start planning now. In Part 2 we will take a look at some of the governance considerations around Algo trading in physical short term power in relation to the effective oversight of this activity.

[1] https://ec.europa.eu/finance/securities/docs/isd/mifid/rts/160719-rts-6_en.pdf

[2] https://www.consilium.europa.eu/en/press/press-releases/2023/11/16/protection-against-market-manipulation-in-the-wholesale-energy-market-council-and-parliament-reach-deal/

[3] Single Intraday Coupling