Browser AI Speedup: 200+ GPU Kernels Generated on the Fly, Not Precompiled
In the browser, there is no 'universally optimal kernel': a kernel is the implementation of an operator on specific hardware, and different devices have different optima. The 200+ kernels are not hardcoded programs but templates that are generated in the browser for your device.
The argument · tap a timestamp to hear it
Operators define, kernels run on hardware
The transcript first distinguishes two concepts: a model is a large graph of operators, and an operator is just a mathematical definition—addition is A+B; a kernel is the implementation that lands that definition on specific hardware. The same add, in the browser, goes through WebGPU/WGSL, while other ecosystems have CUDA or Metal kernels. The two are often conflated, but the speed difference comes from the latter: if the underlying kernel is fast, the whole inference graph can be fast. This also explains why it's worth polishing a kernel for a simple operation rather than vaguely tuning a 'general math library'.
What's released is Jinja templates, not hardcoded kernels
The difference in approach is worth noting: the repo doesn't contain hardcoded WGSL files one by one, but several Jinja templates that use if/else to describe branches for different devices. At execution, the library validates the template, selects the appropriate data type and workgroup size, and renders the WGSL most suitable for the current GPU adapter to run. For example, the same addition function: on the left, the data type is narrow and the workgroup is small; on the right, because the device supports wider data types and larger workgroups, the same add runs faster. The official release lists 200+ kernel entries, but what's really delivered is the ability to 'make kernels on the fly for the hardware'.
Attention in the browser takes only about 20 lines of JS
The demo brings abstraction back to reality: running a small attention mechanism in the browser required only about 20 lines of JavaScript. The flow is text→token→vector, but a token's vector can't be viewed in isolation; you need to see how other tokens in the context affect it. The code just pulls in four kernels—matmul, softmax, add, layer normalization—from the Hub; the rest of the scheduling and GPU data exchange is hidden by the Hugging Face Kernels library. The heavy lifting is all at the kernel layer, which is why the barrier to 'local inference in the browser' has dropped.
The entire wave animation is just one matrix multiplication
The same demo gives an intuitive comparison: 128 waves pass through a 1024×1024 matrix, and each cell's color is an RGBA computed by a matrix multiplication. The animation of over a million pixels runs at about 60fps under WebGPU—60 is not the compute limit, just requestAnimationFrame locking rendering to 60; the kernel itself can compute at about 400fps. Running the same logic in plain JavaScript yields roughly 6.75fps. The gap isn't in the math, but in whether someone has written an implementation that hugs this particular GPU—that's what those kernels do.
Fleet turns user devices into tuning data via tests
Fleet is ostensibly a benchmark page: anyone can run their own GPU and see how it performs in comparison. But the team is blunt: they only have a few devices on hand, while the real world has thousands of WebGPU devices, possibly with edge cases they haven't thought of—some kernels might perform poorly on your machine or even fail outright. Fleet collects runtime data from real devices to continue optimization, and the improved kernels, being open source, return to all users. Clicking 'Run Benchmark' isn't just about seeing a score; you're also helping the project run a round of real-world adaptation testing.
In their own words · checked verbatim
the kernel is the implementation that can actually run the operation.
we don't ship WGSL files. We do ship Jinja files, so Jinja templates.
we basically have implemented an attention mechanism in pure JavaScript.
We only need one kernel. We only need the matrix multiplication.
All of that data is so valuable for us.
Figures
| Number of WebGPU kernels released | 200+ | 0:03 |
| Number of encoding units in the opening animation | about 8000 | 0:03 |
| Lines of JS needed to implement attention | about 20 | 7:12 |
| Wave animation matrix size | 1024 rows × 1024 columns / 128 waves | 8:12 |
| Wave animation pixel count | over 1 million | 8:12 |
| Actual GPU frame rate | about 60 fps (limited by requestAnimationFrame) | 8:12 |
| Theoretical GPU frame rate | about 400 fps | 9:14 |
| Frame rate for the same task in pure JavaScript | about 6.75 fps | 9:14 |
Glossary
- kernel
- The implementation of an operator on specific hardware; the same addition needs different kernels on different GPUs.
- WGSL
- WebGPU Shading Language, the language for writing kernels in the WebGPU environment; it's the bridge to the GPU in the browser.
- Jinja
- A Python template engine; HF uses .jinja files to describe kernels, then renders them to WGSL at runtime based on the device.
- WebGPU
- The next-generation API for accessing the GPU from the browser; the computational foundation for local AI inference on the browser side.
- requestAnimationFrame
- The browser's loop callback that drives animation frames, typically capping the frame rate at about 60fps.
How to listen
Engineers running LLMs in the browser or building high-performance front-end AI applications, and technical decision-makers interested in WebGPU and WASM local inference solutions.
The code walkthrough in the opening demo and the release channel introduction at the end can be sped through; focus on the kernel templating mechanism and the Fleet data feedback section.