umarjamilai

Derives and hand-codes distributed training, Flash Attention and other core algorithms; code-level, reproducible and refutable.

Deep reads here1 / 15 episodes

Cadence~1 every 46.9 days

Latest2026-08-18

TopicsAI & Tech

PriorityT3

19:35:12

umarjamilai 0818

The real culprit behind LLM inference bottlenecks is not compute capacity — it's GPUs waiting for data

H100's compute is 600x its memory bandwidth; during decoding, each step produces just one token but must fetch the entire KV cache, so GPUs spend most of their time waiting for data transfer, not computing.

8 Points 7 Quotes distributed trainingGPU parallelism