AI-Chip & ML-Systems Engineering (GPU Kernels to Compilers)
Build the layer that makes AI fast on hardware — the layer of the AI-infrastructure stack that sits closest to the hardware.
Build the layer that makes AI fast on hardware — the layer of the AI-infrastructure stack that sits closest to the hardware. You start with just enough deep learning to know what you're optimizing (tensors, GEMM, the transformer, attention, what's memory- vs compute-bound, quantization), then engineer the GPU kernels behind modern models: optimized matmul and tensor cores, softmax/LayerNorm reductions, and kernel fusion — culminating in a FlashAttention-style fused attention kernel. You then rise above hand-tuned CUDA to the layer that survives automation: Triton (block-level GPU programming in Python) and ML compilers (MLIR, TVM, torch.compile) that lower a model graph to hardware. The capstone is a fused attention kernel measured against a vendor baseline. This is Branch B of the Zero-to-Low-Level-Systems-Elite program — the skills used in GPU-kernel, ML-compiler and AI-accelerator engineering, where the lasting skill is designing the kernels and compilers, not memorizing intrinsics a model can emit.
6 phases · 48 projects · v1.1.0
Target roles: GPU Kernel Engineer, ML Systems / Inference Engineer, ML Compiler Engineer, AI-Chip / Accelerator Engineer
Stack: CUDA, C++17/20, Triton, Python / PyTorch, MLIR / TVM
PHASE 1 · 8 projects
Deep Learning for Systems Engineers
Learn what the hardware actually runs: tensors and GEMM, the transformer and attention, training vs inference, memory- vs compute-bound ops, and numeric precision/quantization.
01 · featured
Tensors & the Shape of Computation
Explore tensors as N-dimensional arrays with shape, strides, and dtype; understand row-major layout, contiguity, and broadcasting, and connect each concept to its GPU memory-access implication.
Python · NumPy · PyTorch
02 · featured
GEMM: The Operation That Runs AI
Implement matrix multiply from scratch, count FLOPs using the 2×M×N×K formula, measure GPU GEMM throughput, and identify which transformer layers dominate the FLOPs budget.
Python · NumPy · PyTorch
03 · featured
A Neural Net Forward Pass from Scratch
Implement a 3-layer MLP forward pass in NumPy, instrument it with FLOPs and bytes-per-layer counts, classify each op as compute-bound or memory-bound, and verify against PyTorch's nn.Sequential.
Python · NumPy · PyTorch
04 · featured
Anatomy of a Transformer
Implement a full transformer encoder layer from scratch in PyTorch, produce a per-component FLOPs breakdown, compute weight and activation memory, and map every sub-block to its kernel type (GEMM vs elementwise).
Python · PyTorch
05 · featured
The Attention Mechanism in Detail
Implement scaled dot-product and multi-head attention from scratch in NumPy and PyTorch, apply causal masking, quantify the O(S²) memory cost, and compare standard attention to F.scaled_dot_product_attention.
Python · NumPy · PyTorch
06 · featured
Compute-Bound vs Memory-Bound Ops
Build a roofline calculator, compute arithmetic intensity for every major ML op, classify each as compute-bound or memory-bound, and produce a per-op optimization agenda for a full transformer layer.
Python · NumPy · PyTorch
07 · featured
Numeric Precision & Quantization
Explore fp32/fp16/bf16/int8 numeric formats, implement symmetric int8 quantization, benchmark GEMM throughput across dtypes on GPU, and implement a mixed-precision forward pass with torch.autocast.
Python · NumPy · PyTorch
08 · featured
Profile a Model's Forward Pass
Build a decoder-only mini transformer, profile its forward pass with the PyTorch profiler, compute MFU, classify the top bottlenecks using the roofline model, and produce a written optimization agenda for Phase 2.
Python · PyTorch
PHASE 2 · 8 projects
GPU Kernel Engineering for ML
Write the kernels behind deep learning: optimized GEMM and tensor cores, elementwise/activation kernels, and reductions like softmax and LayerNorm — analyzed with the roofline.
01 · featured
Elementwise & Activation Kernels
Write ReLU, GELU, sigmoid, add, and scale kernels using grid-stride loops and float4 vectorization, then benchmark each in GB/s against the GPU's peak memory bandwidth to confirm they are bandwidth-limited.
CUDA · C++17 · nvcc · cudaEvent_t
02 · featured
GEMM for ML
Count FLOPs for real transformer layers, implement a tiled GEMM kernel benchmarked in GFLOP/s vs the GPU peak, and compare against cuBLAS to understand what register tiling, double-buffering, and tensor cores contribute.
CUDA · C++17 · nvcc · cuBLAS · cudaEvent_t
03 · featured
Tensor Cores & Mixed Precision
Write a minimal Tensor Core GEMM using nvcuda::wmma (FP16 inputs, FP32 accumulate), measure the TFLOP/s multiple over FP32 CUDA cores, and compare to cuBLAS HGEMM to understand what the remaining gap requires.
CUDA · C++17 · nvcc · nvcuda::wmma · CUDA FP16 · cuBLAS
04 · featured
Softmax — A Reduction Kernel
Implement a numerically stable, block-parallel softmax kernel using shared memory tree reduction and warp shuffles, benchmark it in GB/s, and validate output against a CPU reference to understand the reduction pattern that underlies FlashAttention.
CUDA · C++17 · nvcc · cudaEvent_t · warp shuffles
05 · featured
LayerNorm & Normalization Kernels
Implement LayerNorm using two-pass (mean then variance), Welford one-pass, and fused affine variants, plus RMSNorm, measuring bandwidth improvement across implementations and validating correctness with identity affine parameters.
CUDA · C++17 · nvcc · cudaEvent_t · shared memory reduction
06 · featured
Memory Optimization for ML Kernels
Apply coalescing fixes, shared memory staging, float4/half2 vectorized loads, and occupancy tuning to ML kernels, measuring the bandwidth improvement from each technique and assembling them into an optimized LayerNorm.
CUDA · C++17 · nvcc · float4 · half2 · cudaOccupancyMaxActiveBlocksPerMultiprocessor
07 · featured
Roofline Analysis of ML Kernels
Compute arithmetic intensities for all Phase 2 kernels, place them on the GPU roofline model, run Nsight Compute on at least one kernel, and produce an optimization priority matrix based on each kernel's distance from its theoretical ceiling.
CUDA · C++17 · nvcc · Nsight Compute · roofline model
08 · featured
An ML Kernel Mini-Library
Assemble optimized ReLU, GELU, softmax, LayerNorm, RMSNorm, and tiled GEMM kernels into a structured CUDA library with a clean C++ header interface, a correctness test suite against CPU references, and a benchmark table reporting each kernel's % of theoretical peak bandwidth or compute.
CUDA · C++17 · nvcc · cudaStream_t · cudaEvent_t
PHASE 3 · 8 projects
Kernel Fusion & FlashAttention
Beat the memory wall with fusion: why fusion wins, fusing elementwise+reductions, online softmax, and a tiled FlashAttention-style fused attention kernel that never materializes the score matrix.
01 · featured
Why Fusion Matters
Measure the HBM round-trip penalty of an unfused 3-kernel pipeline vs a fused kernel and explain the memory wall that kernel fusion addresses.
CUDA · C++ · nvcc · Nsight Systems · NVTX
02 · featured
Fusing Elementwise Chains
Fuse bias-add + GELU + scale into a single CUDA kernel, eliminate intermediate HBM round-trips, and measure the bandwidth and latency win on transformer FFN-scale tensors.
CUDA · C++ · nvcc
03 · featured
Fusing Reductions with Elementwise
Fuse LayerNorm's two-pass reduction+normalization into a single kernel using shared memory, eliminating one full HBM read of the input tensor.
CUDA · C++ · nvcc
04 · featured
Online (Streaming) Softmax
Derive and implement the running-max / running-sum rescaling trick that computes numerically stable softmax over blocks without seeing all elements first — the mathematical core of FlashAttention.
CUDA · C++ · nvcc
05 · featured
Naive Attention on the GPU
Implement 3-kernel naive attention (QK^T → softmax → V), measure the O(N²) score-matrix memory footprint, and observe the GPU OOM boundary that motivates FlashAttention.
CUDA · C++ · nvcc
06 · featured
The FlashAttention Idea
Study and implement the IO-aware FlashAttention algorithm: tile Q/K/V to fit in SRAM, use online softmax running stats, and accumulate output without ever materializing the N×N score matrix.
CUDA · C++ · nvcc
07 · featured
Implementing a Fused Attention Kernel
Implement a working tiled fused single-head attention CUDA kernel using shared memory tiles and the online softmax recurrence, verify against naive attention, and measure peak memory and latency.
CUDA · C++ · nvcc
08 · featured
Fused Attention Benchmark Lab
Benchmark fused vs naive attention across sequence lengths 512–8192, measure O(N) vs O(N²) HBM scaling empirically, profile with Nsight Compute, and produce a capstone results table.
CUDA · C++ · nvcc · Nsight Compute
PHASE 4 · 8 projects
Triton — GPU Programming in Python
Program GPUs at the block level in Triton: the programming model, writing vector-add/softmax/matmul kernels, autotuning, and why Triton sits above hand-tuned CUDA.
01 · featured
Triton — GPU Programming in Python
Understand the motivation for Triton (the CUDA productivity wall), the block-level programming model, and set up a working Triton environment on Colab; run a minimal @triton.jit kernel that introduces program_id, tl.arange, tl.load, tl.store, and the launch grid.
Python · Triton · PyTorch · CUDA · Colab
02 · featured
Your First Triton Kernel — Vector Add
Implement element-wise vector addition in Triton end-to-end: write the @triton.jit kernel with tl.program_id / tl.arange / tl.load / tl.store / mask, write the Python driver, verify against PyTorch for both power-of-2 and non-power-of-2 sizes, and benchmark bandwidth against torch.add.
Python · Triton · PyTorch · CUDA
03 · featured
Fused Softmax in Triton
Implement a row-wise numerically stable softmax kernel in Triton (one program per row, in-kernel tl.max and tl.sum reductions, BLOCK_SIZE = next_power_of_2(n_cols)), verify against torch.softmax, and benchmark to demonstrate 2.5–3x speedup over naive unfused PyTorch.
Python · Triton · PyTorch · CUDA
04 · featured
Matrix Multiply in Triton
Implement a tiled float32 GEMM in Triton: 2D program grid with pid_m/pid_n, tl.dot for tile MAC, accumulator register tile, pointer arithmetic with 2D masking; verify against torch.matmul and benchmark TFLOP/s toward cuBLAS-class throughput.
Python · Triton · PyTorch · CUDA
05 · featured
Autotuning Kernels
Add @triton.autotune with a 12-config search space (BM/BN/BK × num_warps × num_stages) to the matmul kernel, observe the autotuner's first-call search and cache behavior, inspect the winning config, and measure 20–50% TFLOP/s improvement over the fixed-config baseline.
Python · Triton · PyTorch · CUDA
06 · featured
Fused Attention in Triton
Implement a FlashAttention-style single-headed forward attention kernel in Triton: tiled Q/K/V loads, online softmax update (running max + running sum rescaling), running output accumulator; verify against a reference torch implementation and benchmark against naive three-op PyTorch and F.scaled_dot_product_attention.
Python · Triton · PyTorch · CUDA
07 · featured
Triton vs CUDA — The Tradeoffs
Conduct a structured productivity and performance comparison between Triton and CUDA across four implemented algorithms, analyze Triton's expressibility limits, observe Inductor-generated Triton via torch.compile, and articulate a personal decision framework for when each tool is appropriate.
Python · Triton · PyTorch · CUDA · torch.compile · Inductor
08 · featured
Triton Kernel Lab — Phase 4 Finale
Implement and autotune a fused LayerNorm forward kernel in Triton (one program per row, tl.sum for mean and variance, in-register normalization and affine transform), verify against torch.layer_norm, and produce a three-way benchmark table comparing Triton vs torch.layer_norm vs naive 4-op PyTorch.
Python · Triton · PyTorch · CUDA
PHASE 5 · 8 projects
ML Compilers — MLIR, TVM & Graph Optimization
Move up to the compiler layer: computational graphs, operator fusion and lowering, MLIR dialects, TVM and torch.compile, and how a model becomes hardware instructions.
01 · featured
Computational Graphs
Trace a PyTorch model to a torch.fx graph and inspect every node, edge, and data-flow dependency — learning why the graph representation is the foundation that makes all ML compiler optimization possible.
Python · PyTorch · torch.fx
02 · featured
Graph-Level Optimizations
Implement Dead-Code Elimination, Common-Subexpression Elimination, and constant folding as torch.fx graph passes, then demonstrate operator fusion — reducing a 7-node raw graph to 4 nodes while preserving exact semantics.
Python · PyTorch · torch.fx
03 · featured
torch.compile & TorchInductor
Compile a transformer FFN block with torch.compile, measure the eager-vs-compiled speedup, inspect the generated Triton kernel via TORCH_LOGS, and diagnose and fix a graph break — learning the full torch.compile production workflow.
Python · PyTorch · TorchInductor · Triton
04 · featured
MLIR: The Multi-Level IR
Learn MLIR's dialect system and progressive lowering by tracing a matrix multiply from the high-level linalg dialect through affine loops, GPU launch ops, and NVVM down to PTX — understanding why every serious ML compiler (Triton, IREE, XLA) is built on this infrastructure.
Python · PyTorch · MLIR · Torch-MLIR
05 · featured
Apache TVM & Auto-Scheduling
Define a fused ReLU+scale operation as a Tensor Expression, write naive and optimized schedules (fused + vectorized + parallel), measure the speedup, and understand how TVM's Ansor auto-scheduler searches the schedule space to target any hardware from one model description.
Python · PyTorch · TVM
06 · featured
Lowering & Code Generation
Trace the complete lowering pipeline from a PyTorch matmul and softmax through TorchInductor's Triton codegen, illustrative PTX, and down to GPU SASS — understanding what performance decisions are made at each lowering stage and why tiling is the central one.
Python · PyTorch · Triton · MLIR
07 · featured
The Compiler Layer as the Durable Edge
Analyze the automation trajectory from hand-written CUDA to compiler-generated kernels, map the skills required at the largest AI and platform companies roles to your current capabilities, and produce a concrete 6-month roadmap toward a role at the compiler/architecture layer.
Python
08 · featured
Compiler Optimization Lab
Take a transformer block (self-attention + FFN), run it eager vs torch.compile vs max-autotune, capture the generated Triton kernels, identify every fusion decision, quantify memory bandwidth savings, and produce a complete compiler analysis report — a portfolio-quality artifact demonstrating Phase 5 mastery.
Python · PyTorch · TorchInductor · Triton
PHASE 6 · 8 projects
Inference Optimization & the Fused-Attention Capstone
Optimize real inference (KV-cache, quantization, batching, serving engines) and build the capstone: a fused attention kernel measured against a vendor baseline.
01 · featured
Inference vs Training
Quantify the memory, compute, and bandwidth constraints that make inference a distinct engineering discipline from training, measuring TTFT and TPOT on a small decoder-only transformer and deriving why decode is memory-bandwidth-bound.
Python · PyTorch · the open model hub Transformers · Roofline Analysis
02 · featured
The KV-Cache
Implement multi-head attention with and without KV-cache in PyTorch, verify numerical equivalence, and compute the memory footprint of the cache for production LLMs to understand why PagedAttention was necessary.
Python · PyTorch · LLM Inference · Memory Analysis
03 · featured
Quantization for Inference
Implement INT8 and INT4 group quantization from scratch, measure quantization error vs FP16 reference, compute the bandwidth win during memory-bound decode, and understand GPTQ and AWQ's activation-aware strategies.
Python · PyTorch · Quantization · GPTQ · AWQ
04 · featured
Batching & Serving
Simulate static vs continuous batching schedulers, derive the arithmetic intensity vs batch size curve, measure real throughput scaling on a small decoder-only transformer, and build a capacity planning tool showing the compounding benefit of quantization plus continuous batching.
Python · PyTorch · vLLM concepts · Scheduling Simulation
05 · featured
Inference Engines & Runtimes
Profile naive PyTorch inference to identify kernel launch and memory overhead, apply torch.compile to fuse operators, compare TensorRT-LLM, vLLM, llama.cpp, and ONNX Runtime, and build a decision framework for choosing the right engine.
Python · PyTorch · torch.compile · TensorRT-LLM · vLLM · llama.cpp
06 · featured
Optimizing an End-to-End Model
Apply the full inference optimization stack (FP16, torch.compile, INT8 quantization, KV-cache, batching) to a small decoder-only transformer-scale transformer and produce a complete optimization ladder table measuring TTFT, TPOT, throughput, and memory at each stage.
Python · PyTorch · torch.compile · INT8 Quantization · KV-Cache
07 · featured
The AI-Chip Landscape & Your Path
Map the AI-chip ecosystem (NVIDIA, AMD, a hyperscaler TPU, a deterministic-dataflow chip startup, a wafer-scale chip startup, a RISC-V dataflow chip startup, a reconfigurable-dataflow chip startup), understand the CUDA→Triton→MLIR skill ladder, identify related roles and the skills each one asks for, and build a 90-day learning plan.
Python · Industry Landscape · AI Chip Ecosystem · Triton · MLIR
08 · featured
Fused Attention Kernel (Track Capstone)
Implement scaled-dot-product attention in CUDA (naive baseline), then a FlashAttention-style fused tiled kernel using the online softmax algorithm that never materializes the N×N score matrix. Implement a Triton version as the second implementation. Verify numerical correctness against PyTorch's F.scaled_dot_product_attention. Benchmark latency and peak GPU memory across sequence lengths (512–4096) against the naive baseline and PyTorch SDPA. Explain the speedup and memory savings in memory-traffic terms. Package with reproducible setup, correctness test, and a WRITEUP.md with two key tradeoffs.
CUDA C++ · Python · Triton · PyTorch · FlashAttention
Next step
Request AI-Chip & ML-Systems Engineering (GPU Kernels to Compilers)
An advisor reviews the fit and confirms terms in writing. A request is not an acceptance and nothing is charged until you agree.