Browser AI Gets Faster: 200+ GPU Kernels, Not Precompiled but Generated On-Device
There is no universal optimal kernel for AI in the browser: a kernel is an operator's implementation on specific hardware, and different devices mean different optima. The 200+ kernels are not hardcoded programs but templates generated in the browser for your device.
The argument · tap a timestamp to hear it
Operators define, kernels run on the hardware
The transcript first separates two concepts: a model is a big graph of operators, and an operator is just a mathematical definition — addition is A+B; a kernel is the implementation that lands that definition on specific hardware. The same add goes through WebGPU / WGSL in the browser, and corresponds to a CUDA or Metal kernel in other ecosystems. The two are often conflated, but the speed difference comes precisely from the latter: only when the underlying kernel is fast can the whole inference graph be fast. This also explains why it is worth repeatedly polishing a kernel for a simple operation, rather than vaguely tuning a ‘general math library’.
What ships is Jinja templates, not hardcoded kernels
The difference in approach is worth noting: the repository does not hold hardcoded WGSL files one by one, but a number of Jinja templates with if/else branches describing different devices. At execution time, the library validates the template, picks the appropriate data type and workgroup size, then renders it into the WGSL best suited to the current GPU adapter and runs it. For example, the same addition function: on the left the data type is narrow and the workgroup small; on the right, because the device supports a wider data type and a larger workgroup, the same add runs faster. The official release lists 200-plus kernel entries, but what is actually delivered is the ability to make kernels on the fly for the hardware.
Attention in the browser takes only about 20 lines of JS
The demo brings the abstraction back to practice: running a small piece of attention in the browser took only about 20 lines of JavaScript. The flow is text→token→vector, but a token's vector cannot be viewed independently — you have to look at how other tokens in the context affect it. What the code does is merely pull four kernels — matmul, softmax, add, layer normalization — from the Hub; the rest of the scheduling and GPU data exchange is hidden by the Hugging Face Kernels library. The heavy lifting is all at the kernel layer, which is why the barrier to ‘doing local inference in the browser’ has come down.
The whole wave animation is really just one matrix multiplication
The same demo gives an intuitive contrast: 128 waves crossing a 1024×1024 matrix, where the color of every cell is an RGBA computed by a matrix multiplication. The animation of more than a million pixels on screen runs at about 60fps under WebGPU — 60 is not the compute ceiling, it is just requestAnimationFrame locking rendering at 60; the kernel itself can compute at about 400fps. Running the same logic in plain JavaScript leaves the frame rate at roughly 6.75fps. The gap is not in the math but in whether someone wrote it as an implementation hugging this GPU — which is exactly what that batch of kernels is doing.
Fleet turns user devices into a tuning data source via benchmarks
Fleet looks on the surface like a benchmark page: anyone can run their own GPU and see how it compares. But the project team says it plainly: the team only has the few devices on hand, while the real world has thousands upon thousands of WebGPU devices, and there may be edge cases they never thought of — some kernel performs poorly on your machine, or even fails outright. Fleet collects runtime data from real devices back for further optimization, and the optimized kernels return to all users thanks to open source. When you click Run Benchmark once, you are not just looking at a score — you are running a round of real-environment adaptation testing for the project.
In their own words · checked verbatim
the kernel is the implementation that can actually run the operation.
we don't ship WGSL files. We do ship Jinja files, so Jinja templates.
we basically have implemented an attention mechanism in pure JavaScript.
We only need one kernel. We only need the matrix multiplication.
All of that data is so valuable for us.
Figures
| Number of WebGPU kernels released | 200+ | 0:03 |
| Encoding units in the opening animation | about 8000 | 0:03 |
| Lines of JS needed to implement attention | about 20 | 7:12 |
| Matrix size of the wave animation | 1024 rows × 1024 columns / 128 waves | 8:12 |
| Pixels in the wave animation | over 1 million | 8:12 |
| Actual GPU frame rate | about 60 fps (limited by requestAnimationFrame) | 8:12 |
| Frame rate the GPU could theoretically reach | about 400 fps | 9:14 |
| Frame rate of the same task in plain JavaScript | about 6.75 fps | 9:14 |
Glossary
- kernel
- An operator's implementation on specific hardware; the same addition must run a different kernel on different GPUs.
- WGSL
- The language for writing kernels in the WebGPU environment, the bridge for talking to the GPU in the browser.
- Jinja
- A Python template engine; HF uses .jinja files to describe kernels, rendered into WGSL at runtime per device.
- WebGPU
- The new-generation API for browsers to access the GPU, the compute foundation for local AI inference on the browser side.
- requestAnimationFrame
- The browser's loop callback driving animation frames, usually capping the frame rate at about 60fps.
How to listen
Engineers running LLMs in the browser and building high-performance frontend AI applications, plus technical decision-makers watching WebGPU and WASM local inference approaches.
The code walkthrough in the opening demo and the release-channel intro at the end can be sped up; focus on the kernel template mechanism and the Fleet data feedback loop.