aisle

Curriculum · 14 modules · ~62 min

A working knowledge of on-prem AI inference

Learn about AI inference and what makes it a truly unique workload. The modules below build on each other. You can start at the top if you’re new to AI, or jump to a specific module if you know what you’re looking for. By the end you’ll be ready to use the Sizer with confidence and have a productive planning conversation with your AI team.

  1. What is AI inference?

    4 min read

    Inference vs training, where it sits in the AI workload landscape, and why it's the workload most enterprises will run on-prem first.

    Open module

  2. Tokens and context windows

    4 min read

    The unit of text the model reads and writes, and the maximum amount of it the model can hold in mind at once. Every other number in inference is built on these two.

    Open module

  3. How a model makes a token

    4 min read

    The two-phase mechanism that produces each token: prefill (parallel, compute-bound) and decode (sequential, memory-bound), and the TTFT and TPOT timings each phase sets.

    Open module

  4. Inside a GPU

    6 min read

    What a GPU actually is, why HBM holds the model, and why memory bandwidth, not compute, sets the floor on decode latency.

    Open module

  5. The KV cache, the silent capacity killer

    5 min read

    Why each in-flight conversation needs its own working memory, and why context length and concurrency multiply.

    Open module

  6. Managing KV cache pressure

    3 min read

    Three architectural levers (KV cache quantization, MLA, strict context caps) that reduce KV pressure without buying more GPUs.

    Open module

  7. Why your existing playbook breaks

    5 min read

    Four assumptions about capacity planning that don't survive contact with an inference workload.

    Open module

  8. The seven parameters that drive sizing

    8 min read

    The full input set: model size, precision, context, concurrent users, RPS, latency SLOs, burst factor.

    Open module

  9. A worked example, end-to-end

    7 min read

    A regional bank's RAG chatbot: walk through inputs, the math, and the resulting infrastructure recommendation.

    Open module

  10. What an inference server looks like

    1 min read

    The complete picture of a single inference chassis: CPU, RAM, NVMe, NICs, NVLink, power draw, cooling tier, and the facility envelope it has to live in.

    Open module

  11. Beyond one GPU

    1 min read

    Tensor parallelism, pipeline parallelism, expert parallelism, replicas, routing, and the fabrics that hold a multi-GPU or multi-node deployment together.

    Open module

  12. Inference engines

    1 min read

    vLLM, TensorRT-LLM, SGLang, and Triton. What each one is, what they share, and how the engine choice changes the optimization math.

    Open module

  13. Optimization techniques

    7 min read

    The AI team's six big levers. Quantization, batching, caching, LoRA, speculative decoding. What each does, how much it changes the math, when to use it.

    Open module

  14. The IT-and-AI planning conversation

    6 min read

    The curriculum closer. A one-page checklist with role ownership, questions to ask the AI team, red flags to watch for, and the artifacts a successful kickoff produces.

    Open module