Navigating Trade Surveillance Data Challenges

The quality of the alerts generated in a trade surveillance solution is directly correlated with the quality of the data ingested. Obtaining accurate, timely trade surveillance data that reflects the firm’s entire scope of trading activity and position-taking can be a greater challenge than many compliance executives realize initially. 

The data also needs to meet various regulatory obligations. In short, robust data governance is essential for a strong trade surveillance program – reducing false positives and making investigations more efficient. Amazon Web Services (AWS) defines data governance as “a methodology that ensures data is in the proper condition to support business initiatives and operations.” Key elements of data governance include data accuracy, timeliness, source, ownership, security, compliance, and maintenance.

This blog will explore some of the key challenges that compliance teams might face in acquiring the wide range of data points needed for an effective trade surveillance program, and how data governance helps maintain data quality, from data capture to data use.


Trade Surveillance Data Essentials

Compliance teams not familiar with their firm's trading activity or data governance may find trade surveillance requirements overwhelming. A firm's trade data is usually from its Execution Management System (EMS), Order Management System (OMS), or Data Repository – aka, its golden source. It’s critical that this data is of high quality. Poor trading data can lead to more false positives, and potentially undermine the ability to use trade surveillance data as evidence of market abuse.

An effective trade surveillance platform should enrich a firm's trade data with market data from a provider such as Refinitiv. The market data provides information on the security – for example, the daily traded volume, the orders, and the prices which the securities were being traded, opened, or closed at.

Other data is also essential for performing trade surveillance, including:

  • Reference Data – An effective trade surveillance system should resolve the traded securities universal identifiers such as CUSIP, SEDOL, ISIN, and RIC against the market data to normalize it. Firms seeking to use trade surveillance software need to have a good understanding of the kinds of reference data they are using or can acquire.
  • News DataNews articles on people, companies, and the sentiment of the news can help the compliance teams in their investigation, so an effective trade surveillance platform should include news.

SteelEye - Trade Surveillance Data Essentials News
  • Position Data – Knowing a firm or fund holdings of financial assets can provide insight into the trading activities and investigation of alerts raised. Position data typically includes business dates, security descriptions, and a change in the position. The rapid and significant decrease in a position before a drop in security price in context with news and market data may imply that insider information is being used.

Data governance may start with the compliance team's need to implement effective trade surveillance but data governance around accuracy, timeliness, and processing is a firm-wide undertaking for record keeping obligations.

Best Practices for Trade Surveillance Data Collection

The complexities of data aggregation may exist due to multiple execution and order management systems being used, trades being executed on voice channels, and OTC instruments being traded. Key trade surveillance data issues include:

  • Data accuracy and completeness – Trade and position data must be accurate. Firms then need to capture every trade and also multiple data points of the trade, such as the order, the fills, cancellations, amendments, and any other order related data points. Compliance teams must align the data points required with what is needed for the market abuse algorithms to work effectively.
  • Automation tools for data collection  – Using automation tools for data collection is an ideal way to make the data collection process more efficient, and to reduce errors and operational risk. Automation should be implemented within the context of an overall data management strategy.
  • Documenting sources and timestamps – It’s essential that firms map the sources of their trade and position data so there is a good understanding of where the data is coming from and who owns it. This will also help the firm to determine that all trade and position data is being captured. Accurate timestamps are also essential. For example, voice-based salespeople and traders must ensure they are entering their orders and trades into the system as soon as possible. Timestamps must also be for the correct time zone.
  • Trimming and loading the data  – Not all the data that an OMS or EMS produces is relevant for trade surveillance, so firms should remove the data that is not needed. If the firm is sending the data to the trade surveillance software in batches, it needs to make sure the data is error-free before loading it.  
  • Linking the data together  – Firms need to ensure that they have the right security identifiers in place so that the transactional and position data can be connected to other data types by the trade surveillance platform.

SteelEye - Best Practices for Trade Surveillance Data Collection linking the data

For compliance teams, the work of fixing these issues and preventing them from arising again can often be overwhelming. This is why involving the firm’s information technology team early in the process of selecting a trade surveillance solution is advisable. Their expertise can be invaluable. Additionally, the best way to identify potential data issues is to provide a sample of trade and position data to the trade surveillance software vendor as soon as possible. They can then review that sample and discuss with the firm how to resolve data gaps or quality issues.

Utilizing Technology for Trade Surveillance Data Management

Firms should be seeking to automate, when possible, the collecting and uploading of data to the trade surveillance platform. For example, trade surveillance platforms should be able to provide feed handlers, FIX connections, or a place to automatically upload CSV files. Avoiding manual interaction with the data reduces operational risk.

Trade surveillance platforms should provide an easy to use and effective audit of data sent, received, and ingested.  This capability supports continuous improvement of data quality.

Ensuring Trade Surveillance Data Security and Privacy

Trade and position data is highly sensitive because it essentially contains competitive information about a firm’s activities. Therefore, trade surveillance platforms should have data encryption and other robust information security measures in place, such as ISO 27001 certification, and SOC 2 certification.

Firms also need to be sure they are meeting jurisdictional data privacy requirements – recordings of voice calls are usually considered to fall within personal information privacy requirements for rules such as the EU General Data Protection Regulation (GDPR) / Swiss Federal Act on Data Protection (FADP). Also, some data privacy rules require firms to keep the data in the country in which it was created, so trade surveillance needs to be performed locally in the cloud.

SteelEye - Ensuring Trade Surveillance Data Security and Privacy

Trade Surveillance Data Challenges Can Turn Into Fines

Getting the data wrong for trade surveillance can have sizeable consequences for firms – both reputationally and financially. For example, one large international firm was fined £12.6 million in August 2022 by the UK Financial Conduct Authority (FCA) in part because of data failures. According to the Final Notice, the firm “continued to face challenges with order surveillance because of the increased volume of data handling and the different ways in which orders were received, which…was more challenging for certain products.”

A group of brokers was fined £4.8 million by the FCA in December 2022 because they were not capturing voice orders completely – and voice-brokered transactions were more than 80% of their business. Data completeness is as important as data accuracy in the eyes of the regulator.

Building a Robust Trade Surveillance Data Governance Strategy 

Getting trade surveillance data governance right requires compliance teams to be proactive from the very beginning. Taking a strategic approach to managing trade surveillance data will deliver long-term benefits, such as lower false positives and more robust evidence for investigations.

Key elements include:

  • Developing a long-term strategic plan – Take a strategic approach based on data governance principles for the capturing, storing, and sharing trade and position data with the trade surveillance platform.

  • Creating a dedicated architecture – Once a firm decides on its strategy, it should consider what technology infrastructure it will need to invest in to support trade surveillance data governance.

  • Collaboration between departments – Improving collaboration among different teams can help avoid data duplication. For example, having one central team capturing and distributing market data can reduce license costs.

  • Training and education – High quality data starts with people. Provide training so that people understand their data governance and data management responsibilities. This will help to create a data governance culture within a firm.

  • Monitoring and continuous improvement – Data quality is not a one-off project. It’s important that a trade surveillance platform is future-proofed and supports continuous improvement through data analytics and data audit capabilities.

SteelEye - Building a Robust Trade Surveillance Data Governance Strategy

Firms that generate the most benefits from their trade surveillance programs are the ones that take a data governance approach.  Not only will data governance for trade and position data result in reduced false positives and better evidence to support investigations, but it can deliver other benefits too, such as easier compliance with other regulations, better regulatory change management, and improved internal risk and business reporting. 

When a firm is looking to implement a trade surveillance system, it’s important that it thinks about all data sources from day one. Compliance teams should tap into internal expertise – bringing key data, technology, and trading talent to the table early on. Compliance teams should also work with trade surveillance vendors who can bring their own skills to help the firm manage challenges and implement good practices. Over time the trade surveillance platform should support ongoing error identification and data improvement to maximize the benefit of the technology to the whole firm.

How SteelEye can help with Trade Surveillance Data challenges 

SteelEye – a pioneering RegTech provider that offers a comprehensive approach to trade surveillance – has helped many firms get to grips with their trade and position data issues. SteelEye’s team of data experts can:

  • Help compliance teams identify potential data challenges early on and suggest data governance approaches.

  • Analyze a trade data sample provided by the client to potential spot data quality issues.

  • Suggest ways to automate internal trade and position data collection.

  • Enable firms to better understand how regulatory requirements might impact how they collect, store, and use trade surveillance data.


In addition, SteelEye has built data quality and data governance into its integrated surveillance platform through:

  • Deploying high quality market data from Refinitiv.

  • Creating connectors to enable firms to automate the send of order, trade, and position data to our trade surveillance platform.

  • Providing analysis of order, trade, and position data quality through its Insights reporting solution.

  • Delivering audit capability on data sent to the trade surveillance platform for compliance and internal audit purposes.


Turn Supervision into Super Vision

Contact our compliance experts to see our platform in action or learn more about how we can help your firm reduce compliance fatigue.  



Latest News

Navigating AI in Communications Surveillance: Insights from Industry Leaders at Regs & Eggs 2024

| 24 Jun 2024

SteelEye and Intelligent Voice Transform Voice Surveillance through Strategic Partnership

| 18 Jun 2024

Beyond Words: The Hidden Risks of Emojis in Trading

| 06 Jun 2024

Untapped Risk: Why Compliance Needs Voice Transcription

| 22 May 2024

Global Regulators Intensify Scrutiny of Market Manipulation Failures

| 10 May 2024

SteelEye’s Women in RegTech: Sonia Chowdhury

| 09 May 2024