CERN uses ultra-compact AI models on FPGAs for real-time LHC data filtering

TL;DR Highlight

CERN uses a 'hardware-first' inference approach at the LHC by burning PyTorch/TensorFlow models directly into FPGAs to filter hundreds of terabytes of collision data per second at nanosecond latency — a radical departure from conventional GPU/TPU-based AI.

Who Should Read

Embedded/hardware developers working on or considering deploying AI models to edge devices or FPGAs, and ML engineers designing systems that require extreme low-latency (nanosecond to microsecond) inference.

Core Mechanics

The LHC generates approximately 40,000 exabytes of raw data per year (roughly 1/4 of the entire current internet), peaking at hundreds of terabytes per second. Storing or processing all of this is physically impossible, so only about 0.02% of all collision events are preserved.
The Level-1 Trigger, the first filtering stage, consists of approximately 1,000 FPGAs and must evaluate incoming data within 50 nanoseconds. A specialized algorithm called AXOL1TL operates in real time here, instantly determining whether an event is worth preserving.
CERN uses an open-source tool called HLS4ML to convert ML models written in PyTorch or TensorFlow into synthesizable C++ code, which is then deployed directly onto FPGAs, SoCs, and ASICs. This achieves extreme speed with far less power and chip area compared to GPU/TPU-based approaches.
AXOL1TL was trained as a VAE (Variational Autoencoder, a structure that detects anomalies by compressing and reconstructing input data). At inference time, the decoder is removed and only the mu² term of the KL divergence is used as an anomaly score, operating within 2 clock cycles at a 40MHz clock.
A significant portion of chip resources is allocated not to neural network layers but to precomputed lookup tables. By storing results for common input patterns in advance, nearly instantaneous output is possible without floating-point operations. This is the core design philosophy that enables nanosecond-level latency.
Starting from v5, the model was improved by adding a VICREG block and using reconstruction loss, deployed to FPGA via the hls4ml-da4ml flow. Another model, CICADA, was trained as a VAE and then used knowledge distillation with a supervised loss on the anomaly score.
The second filtering stage, the High-Level Trigger, runs on a large computing farm of 25,600 CPUs and 400 GPUs. This stage only activates after the Level-1 Trigger has dramatically reduced the data volume.

Evidence

"One of the authors of the AXOL1TL model commented directly to correct errors in the article, clarifying that the phrase 'burned into silicon' is inaccurate — the model actually runs on FPGAs. While the weights can be considered 'burned in' in the sense that they are hardwired (implemented via shift-add) on the FPGA, the FPGA itself is reprogrammable. Separate projects targeting actual silicon (ASICs), such as SmartPixel and HG-Cal readout, also exist. The same author shared detailed technical specifics: versions prior to v4 were pure VAE-based small MLPs, while v5 introduced VICREG blocks and reconstruction loss. FPGA deployment was implemented using QAT (Quantization-Aware Training) and distributed arithmetic, operating within 2 clock cycles (= 50 nanoseconds) at a 40MHz clock. The community also criticized the article for misusing the term 'LLM' — the system is actually a small VAE-based neural network, not an LLM like ChatGPT. Some argued that even the word 'AI' was an overstatement, suggesting 'a chip with hardcoded logic derived from machine learning' would be more accurate. An interesting discussion emerged about using analog circuits for neural networks, questioning whether matrix-multiplication-based neural networks could yield instantaneous output if implemented in analog. Separately, it was noted that modern CPU branch predictors already use perceptrons, meaning most modern computers already have neural networks embedded in hardware. As a real-world application example, a coffee machine equipped with a tiny CNN model was shared — handling three tasks locally via an onboard camera: cup type classification, cup position image segmentation, and volume regression for coffee quantity adjustment. While not as extreme as CERN, it was cited as another practical example of ultra-small AI models running at the edge."

How to Apply

"If you need to deploy a model trained in PyTorch or TensorFlow onto an FPGA, HLS4ML (https://github.com/fastmachinelearning/hls4ml) can convert your model into synthesizable C++ HLS code. It is the open-source toolchain actually used by CERN and is also applicable to industrial edge AI requiring sub-microsecond latency. When deploying an anomaly detection system in an extremely low-latency environment, you can follow the AXOL1TL approach: train a VAE, then at inference time remove the decoder and use only the mu² term of the KL divergence as the anomaly score. This reduces model size and simplifies computation, making hardware implementation more feasible. If you need to reduce bit-width while maintaining model accuracy for FPGA deployment, apply QAT (Quantization-Aware Training) during the training phase. CERN combined high-granularity quantization with distributed arithmetic to achieve sub-1-microsecond latency; the specific methodology is detailed in the related paper (https://arxiv.org/abs/2405.00645). By replacing a significant portion of neural network operations with lookup tables, you can achieve near-instantaneous responses to recurring input patterns without floating-point arithmetic. This approach is useful not only for FPGAs but also for extremely resource-constrained environments such as embedded MCUs."

Terminology

FPGAShort for Field-Programmable Gate Array — a semiconductor chip that can be freely programmed with any desired digital circuit even after manufacturing. Unlike ASICs (Application-Specific Integrated Circuits), FPGAs are reconfigurable, making them ideal for prototyping and flexible hardware acceleration.

ASICShort for Application-Specific Integrated Circuit — a custom semiconductor designed for a specific purpose. Once manufactured, it cannot be changed, but it is much faster and more power-efficient than an FPGA.

HLS4MLShort for High-Level Synthesis for Machine Learning. An open-source tool that automatically converts PyTorch or TensorFlow models into C++ HLS code deployable on FPGAs/ASICs, enabling ML model deployment to hardware with minimal hardware expertise.

VAEShort for Variational Autoencoder. A neural network trained to compress (encode) and then reconstruct (decode) input data. Because patterns that differ from the training data reconstruct poorly, this property is exploited for anomaly detection.

QATShort for Quantization-Aware Training. A technique that simulates quantization error during training to minimize accuracy loss when reducing model weights from 32-bit floating point to lower bit-widths (e.g., 4-bit, 8-bit).

Level-1 TriggerThe first gate in LHC data filtering. Implemented in FPGA-based hardware, it decides within 50 nanoseconds whether a collision event should be preserved. Data that does not pass this stage is permanently discarded.

룩업 테이블 (Lookup Table)A method of precomputing and storing output values for given inputs in a table. Since only a table lookup is needed rather than actual computation, it is extremely fast. Replacing portions of neural network operations with lookup tables on FPGAs can dramatically reduce latency.