Most "AI-powered trading" companies don't do what quant hedge funds do. They sell dashboards. The top quant firms spend 80% of their effort on data, 15% on infrastructure, and 5% on models. The models are the least differentiated part. That ratio surprises people. It shouldn't.
I've worked in quantitative research for over a decade. The gap between how quant firms are marketed and how they actually operate is wide. This post covers what real ML pipelines look like inside systematic trading firms, profiles the major players, and explains why the talent and data arms races matter more than model architecture. You can browse trading companies in the AIFI directory to see the full landscape.
The marketing gap
Open any fintech pitch deck tagged "AI trading." You'll find candlestick charts, neural network diagrams, and claims about predicting market movements. Most of these products sell technical indicators repackaged with ML labels. Moving averages computed by a random forest are still moving averages.
Real quant firms don't predict prices. They predict small, noisy signals. A good signal might have a Sharpe ratio of 0.3 on its own. Useless in isolation. Valuable when combined with 500 other signals in a portfolio construction framework that manages risk, correlation, and transaction costs simultaneously.
The distinction matters. Price prediction is a solved marketing problem and an unsolved research problem. Signal combination is an engineering problem, and engineering problems have tractable solutions.
The quant ML pipeline
Every serious quant firm runs some version of the same pipeline. The stages are consistent even when the implementations differ.
Data acquisition and engineering. This is where most of the budget goes. Firms ingest traditional market data (prices, volumes, order book depth) alongside alternative data. Alternative data means satellite imagery of retail parking lots, credit card transaction panels from providers like Second Measure, container shipping records, web traffic estimates, app download counts, patent filings, job postings, and anything else that might contain a forward-looking signal about economic activity.
Raw data is messy. Satellite images have clouds. Credit card panels have survivorship bias. Web scraping breaks when sites change layouts. The data engineering teams at top firms are larger than the research teams. That's by design.
Feature engineering. This stage converts raw data into numerical features a model can consume. A parking lot satellite image becomes "week-over-week change in estimated car count at Walmart locations in the Southeast US." A credit card panel becomes "month-over-month spend growth in the restaurant category for consumers aged 25-40."
Domain knowledge matters here more than ML sophistication. A physicist who understands signal processing will build better features from noisy time series than a deep learning expert who doesn't. This is why quant firms hire physicists.
Model training. Gradient boosted trees (XGBoost, LightGBM) still dominate for structured tabular financial data. The reasons are practical: they handle missing values well, they're fast to train, they provide feature importance rankings, and they don't overfit as aggressively as neural networks on small financial datasets. Neural networks get used for unstructured data, NLP on earnings call transcripts, computer vision on satellite imagery, but the final signal combination step usually runs on tree-based models.
Overfitting is the central danger. Financial data has low signal-to-noise ratio. A model that achieves 52% accuracy on historical equity returns is potentially very profitable. A model that achieves 65% accuracy is almost certainly overfit. Firms use walk-forward validation, synthetic data augmentation, and strict out-of-sample testing to manage this. Many promising models die at the validation stage. That's the process working correctly.
Portfolio construction. Individual signals become portfolios through optimization. The optimizer takes hundreds of weak signals, each with estimated alpha, risk, and decay characteristics, and produces target portfolio weights subject to constraints: sector exposure limits, factor exposure limits, turnover limits, position size limits, and transaction cost estimates.
This step is where the math gets dense. Mean-variance optimization, Black-Litterman models, risk parity, and various regularization techniques all appear. The portfolio construction layer is often more important than any individual signal. A mediocre signal in a good optimizer beats a strong signal in a naive portfolio.
Execution. Orders need to reach the market without moving prices. Large firms use execution algorithms that split orders across time and venues to minimize market impact. Reinforcement learning has shown documented improvements here. Man Group/AHL published research showing RL-based execution reduced slippage by measurable amounts compared to traditional TWAP and VWAP algorithms.
The firms
Two Sigma. About $60 billion in AUM. Around 1,800 employees, roughly half of whom are engineers and data scientists. Founded in 2001 by David Siegel and John Overdeck. They describe themselves as a technology company that happens to trade financial markets. Their data infrastructure is their moat. They invest heavily in proprietary data ingestion, storage, and processing systems. Their Venn platform open-sources some of their data science tools.
Jane Street. Primarily a market maker, not a hedge fund. They trade bonds, ETFs, options, and other instruments, providing liquidity and earning the bid-ask spread. Daily notional trading volume exceeds $21 billion. About 2,500 employees. They write most of their production code in OCaml, a functional programming language rarely seen outside academia. Their interview process involves probability puzzles and mental math. They don't manage outside capital.
Citadel Securities. Separate entity from Citadel the hedge fund, though both were founded by Ken Griffin. Citadel Securities handles roughly 25% of US equity volume. They're a market maker and execution firm, using ML for pricing, risk management, and order routing. The distinction between Citadel (hedge fund, ~$65B AUM) and Citadel Securities (market maker) is important and frequently confused.
Bridgewater Associates. The largest hedge fund by AUM at over $120 billion. Founded by Ray Dalio in 1975. Their approach is systematic macro: they build models of how economies work and trade based on those models. "Pure Alpha" is their flagship strategy. "All Weather" is their risk parity strategy. Bridgewater's ML adoption has been more gradual than the tech-native firms. They've historically relied more on economic reasoning than statistical pattern recognition.
Man Group / AHL. The most transparent quant firm when it comes to ML research. Their research team publishes papers regularly. Notable work includes RL for execution optimization, transfer learning across markets, and portfolio construction under transaction costs. AHL manages about $50 billion. Their willingness to publish gives outside researchers an unusual window into production quant ML.
WorldQuant. Founded by Igor Tulchinsky, a former DE Shaw quant. WorldQuant's model is distinctive: they crowdsource alpha research from over 600 researchers globally. They evaluate millions of alpha signals through an automated backtesting platform. Individual signals are weak. The portfolio-level combination is where the value sits. This approach trades signal quality for signal quantity and diversity.
Numerai. A hedge fund structured as a data science tournament. External participants build models on obfuscated data and stake cryptocurrency (NMR tokens) on their predictions. Numerai combines the best submissions into a meta-model that trades equities. It's an experiment in decentralized alpha generation. Whether it produces competitive risk-adjusted returns over long periods remains an open question. The structure is genuinely novel.
QuantConnect. An open-source algorithmic trading platform with over 200,000 users. Their Lean engine lets individuals and small firms build, backtest, and deploy systematic strategies. They sit at the infrastructure layer rather than the fund layer. QuantConnect demonstrates how commoditized the tools have become. The tools aren't the bottleneck. The data and capital are.
Talent
The competition for quantitative researchers is intense. Top firms pay $400,000 or more as starting compensation for PhD graduates in math, physics, statistics, or computer science. Senior quant researchers at firms like Jane Street or Citadel can earn well into seven figures.
Firms recruit heavily from a small set of programs: MIT, Stanford, Princeton, CMU, Caltech, Cambridge. The competition with big tech (Google, Meta, OpenAI) for the same talent pool is real and ongoing. Firms have responded by increasing compensation, offering more intellectually stimulating research problems, and, in some cases, allowing researchers to publish (Man Group being the clearest example).
The skillset has shifted. Twenty years ago, quant firms wanted financial engineers who could price derivatives. Now they want ML engineers who can build data pipelines and train models at scale. The financial knowledge is still necessary but no longer sufficient.
The alternative data arms race
Alternative data is where the real competition happens. When every firm uses the same market data and similar models, the edge comes from having data nobody else has.
Satellite imagery providers like Orbital Insight and Planet Labs sell data that quant firms use to estimate economic activity. How many cars are in Walmart's parking lots? How much crude oil is sitting in storage tanks? How many shipping containers are stacked at the Port of Shanghai? These measurements, updated daily or weekly from space, provide information that won't appear in official statistics for months.
Credit card transaction data from providers like Second Measure and Earnest Research gives near-real-time revenue estimates for public companies. App usage data from Sensor Tower and Apptopia provides user engagement metrics before quarterly earnings reports. Web scraping captures pricing changes, job postings, product listings, and customer reviews across millions of sites.
The problem with alternative data is decay. Once a dataset becomes widely available, its alpha erodes. Satellite imagery of parking lots was a meaningful edge in 2015. By 2025, most major quant firms had access to the same providers. The arms race moves to newer, less commoditized data sources. Firms that can find and process novel data fastest win.
What this means
The quant industry runs on ML. That's not debatable. But the ML that matters looks nothing like the marketing. It's data cleaning pipelines, feature engineering by domain experts, gradient boosted trees on tabular data, and portfolio optimizers that combine hundreds of weak signals under risk constraints.
The firms that win do three things well: they acquire data others don't have, they hire researchers who can extract signals from that data, and they build infrastructure that lets those researchers iterate fast. Models are interchangeable. Data and talent are not. Anyone selling "AI trading" without a clear data advantage and a research team with quantitative credentials is selling something else.