Compute is the new capital: why AI startups raise GPUs, not just cash
For AI startups, access to GPUs can matter more than money. How cloud deals, credits and chip reservations shape who can build frontier models.
AI Desk3 min read
Tag
#Gpus
Every The Latent Daily story, tool, model and resource tagged “Gpus”.
4 items
How LLM inference serving works: batching, KV cache and vLLM
What happens when a server answers thousands of LLM requests: prefill and decode, the KV cache, continuous batching, PagedAttention and the metrics that matter.
AI Desk3 min read
How much GPU memory does an AI model need? A sizing guide
Estimate the GPU memory needed to run or fine-tune a language model: weights, KV cache, activations and overhead, with worked examples for 8B and 70B models.
AI Desk4 min read
GPU vs TPU vs NPU: the AI chips explained
Why AI runs on GPUs, what Google's TPUs and the NPUs in laptops and phones do differently, and how memory, bandwidth and interconnects shape AI performance.
AI Desk3 min read