Back to home

Scaling LLMs: Systems, Optimization, and Emerging Paradigms

COMS E6998, taught by Dr. Kaoutar El Maghraoui

This graduate research seminar covers how large language models are trained, served, and run on hardware. Students read 24 core papers over twelve weeks, starting with scaling laws and ending with wafer-scale inference. Each paper is presented by one student team and critiqued by another, and each group builds a survey backed by its own experiment.

What the seminar covers

The questions about large language models in 2026 are mostly about systems and cost: how a frontier model is trained without wasting compute, how it is served quickly and cheaply, and how it learns to reason and to act. The seminar covers each of these in turn, starting with scaling laws and ending with the hardware. Every week comes back to hardware and software co-design, because most of the efficiency gains in these papers come from it.

The reading list is recent and includes many industry papers. Production systems from ByteDance, Moonshot AI, DeepSeek, Qwen, and Groq are read together with research papers from OSDI, NSDI, MLSys, and ICLR. Students leave able to read a systems paper critically and reason about cost and latency.

The seminar assumes High-Performance Machine Learning or equivalent background in compression, quantization, distributed training, FlashAttention, speculative decoding, and vLLM serving, so it starts at the current research frontier.

Dr. Kaoutar El Maghraoui presenting the slide Five ways we scale an LLM system in 2026, covering capacity, context, inference compute, workflow, and infrastructure, in the Scaling LLMs seminar at Columbia University.
The first session lists five ways an LLM system scales in 2026, from capacity to infrastructure, and the bottleneck each one creates.
Dr. Kaoutar El Maghraoui pointing at a roofline chart of prefill and decode during the Scaling LLMs seminar at Columbia University.
In one session, I show where prefill and decode fall on the roofline.
Students in the Scaling LLMs seminar at Columbia University during a session.
Students follow a seminar session.

How the seminar is organized

The semester moves through five content modules, and a final project runs alongside them. Every week follows the same pattern: two core papers, each with a presenter team and a discussant team.

Semester arc. Week 1 is orientation. Five module cards follow in sequence: I, foundations and architecture, Weeks 2 to 4; II, inference and economics, Weeks 5 to 8; III, reasoning and alignment, Weeks 9 to 10; IV, agents and multimodal, Weeks 11 to 12; V, hardware futures, Week 13. Below them, the final project runs from Week 3 to Week 14 with a proposal in Weeks 3 to 4, a progress check around Week 8, and presentations in Week 14.
The modules cover training and architecture first, then inference, reasoning, agents, and hardware. The final project runs from Week 3 to the last day of classes.
I

Foundations and Architecture

Weeks 2 to 4

The module covers scaling laws that account for inference cost and distributed training on more than 10,000 GPUs. It then turns to architectures that replace the uniform dense Transformer with sparse mixture-of-experts, hybrid attention and recurrent layers, and hardware-aligned sparse attention.

II

Inference Optimization and Economics

Weeks 5 to 8

Most production cost is in inference, so this module spends four weeks on it: disaggregated serving and SLO-aware scheduling, long-context KV memory, INT4/KV4/FP4 co-design, and speculative and diffusion-based decoding. Each method is measured by goodput, tail latency, memory traffic, and dollars per token.

III

Reasoning and Alignment

Weeks 9 to 10

The first week covers RL post-training and the rollout bottlenecks it creates. The second covers test-time compute, which asks how an inference system should spend a token and latency budget on a reasoning model.

IV

Agentic AI and Multimodal

Weeks 11 to 12

The module covers how long-horizon agents are evaluated and how agent workflows are scheduled onto models and hardware. It ends with multimodal serving, where one request can pass through a graph of separate model components.

V

Hardware Futures

Week 13

The module compares GPU clusters, wafer-scale systems, SRAM-based deterministic pipelines, TPUs, and other inference accelerators by asking which bottleneck each design removes and which one it creates.

VI

Final Project

Weeks 3 to 14

Each group writes a survey and runs an experiment that tests a concrete scaling claim or trade-off. The work runs across the semester and is presented on the last day of classes.

Weekly session. Before class, everyone reads both core papers and the assigned teams add a supplementary paper. The session runs two hours: 15 minutes of instructor framing, then for each paper a 20-minute presenter talk and a 20-minute critique and Q&A, then about 20 minutes of open discussion and five minutes of logistics. After class there is a quiz and an optional lab notebook.
Each session starts with a short framing deck. Then each core paper gets a 20-minute talk, followed by a critique and Q&A.

Core and supplementary papers

Each content week has two core papers that everyone reads and that the weekly quiz covers. Supplementary papers give history, competing approaches, or newer work. The presenter and discussant teams that week read at least one supplementary paper for each core paper.

Presenter and discussant teams

Every student presents one paper in a team and, in a different week, serves on the discussant team for another paper. Each core paper has its own presenter team and discussant team, so every student takes each role once. Presenters give a 20-minute talk on the problem, method, results, and limitations. Discussants give a 10-minute critique, lead the discussion, and submit a short write-up.

Weekly quiz

A short online quiz on the week's two core papers follows each session. One or two questions can come from concepts in that week's notebook.

Hands-on notebooks

Each content week has a short notebook that makes the idea concrete, for example measuring a KV cache or watching speculative decoding accept and reject tokens. Notebooks are not submitted or graded, and most run on a free Colab GPU or a CPU.

25%
Weekly quizzes
35%
Paper presentation and discussion
40%
Final project, up to 10 bonus points

The papers we study

The list has 24 core papers, two per content week, each with a short summary. Each week also lists supplementary readings (38 in total) that give context and project ideas. I update the list as new work appears.

Week 1Course introduction and the current LLM landscape
I lead this session on what has changed since HPML, including reasoning models, agents, multimodal models, and the economics of inference. We also go over course mechanics and how to read a systems paper, and the projects start.
Week 2, Module IScaling laws after Chinchilla: training, architecture, and inference cost
Training Compute-Optimal Large Language Models (Chinchilla)
Hoffmann et al., NeurIPS 2022
The authors train more than 400 language models, from 70M to 16B parameters on 5B to 500B tokens, to find how model size and training data should grow under a fixed compute budget. They conclude that both should scale equally. Chinchilla (70B parameters, 4× more data than the 280B Gopher at the same compute) outperforms Gopher, GPT-3, and Megatron-Turing NLG.
Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs
Bian et al., ICLR 2026
This paper adds architecture to the Chinchilla scaling law. A conditional scaling law fitted over more than 200 models (80M to 3B parameters) shows how hidden size, the MLP-to-attention parameter ratio, and grouped-query attention affect both accuracy and inference cost. With the same training budget, the architectures it selects reach up to 2.1% higher accuracy and 42% higher inference throughput than LLaMA-3.2.
Supplementary readings (3)
Week 3, Module ITraining at frontier scale: parallelism, heterogeneity, and communication
MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
Jiang et al., NSDI 2024
MegaScale is ByteDance's production system for training LLMs on more than 10,000 GPUs. The team co-designed the model block and optimizer, overlapped communication with computation, and tuned operators, the data pipeline, and the network, then built diagnosis tools to find failures and stragglers. Training a 175B model on 12,288 GPUs, the system reaches 55.2% MFU, 1.34× higher than Megatron-LM.
Tessera: A Holistic Pipeline Parallelism Framework for Trillion-Parameter Heterogeneous MoE Training
Hu et al., OSDI 2026
Pipeline systems assume uniform layers, but models such as Qwen3 mix sparse MoE layers with different attention variants, so stages become unbalanced once communication overlaps computation. Tessera schedules the overlap for each layer combination, partitions the pipeline from measured post-overlap costs, and fills MoE-routing stalls at runtime. In Qwen3 pre-training on 4,096 to 12,288 GPUs it raises throughput by 20% to 33% over the production baseline.
Supplementary readings (3)
Week 4, Module IBeyond the dense Transformer: sparse MoE and hybrid sequence models
DeepSeek-V3 Technical Report
DeepSeek-AI, 2024
DeepSeek-V3 is a 671B-parameter mixture-of-experts model that activates 37B parameters per token. It combines Multi-head Latent Attention, DeepSeekMoE with auxiliary-loss-free load balancing, and a multi-token prediction objective, and it was trained in FP8 mixed precision on 14.8T tokens. The full training run took 2.788M H800 GPU hours.
Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance
Zuo et al., 2025
Falcon-H1 runs Transformer attention and state-space (Mamba) heads in parallel inside each layer and is released from 0.5B to 34B parameters. The report revisits model design, data strategy, and training dynamics. The 34B instruction-tuned model matches or beats models up to 70B, such as Qwen2.5-72B and Llama-3.3-70B, at about half their size.
Supplementary readings (3)
Week 5, Module IIProduction serving: disaggregation, scheduling, and SLO-aware goodput
Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
Qin et al., FAST 2025
Mooncake is the serving platform behind Moonshot AI's Kimi. It separates prefill and decode clusters and builds a disaggregated KV cache from the cluster's underused CPU, DRAM, and SSD capacity. Its scheduler maximizes throughput under latency SLOs and rejects requests early when the system is overloaded. On real workloads it lets Kimi handle 75% more requests.
JITServe: SLO-aware LLM Serving with Imprecise Request Information
Zhang et al., NSDI 2026
Requests now carry different latency goals: per-token latency for streaming chat, full-response deadlines for tool calls, and chains of dependent calls for agents. JITServe schedules with imperfect estimates of response length and dependencies, refines them as generation proceeds, and gives each request just enough capacity to meet its SLO. It improves goodput by 1.4× to 6.3× over prior schedulers, or uses 28.5% to 83.2% fewer resources.
Supplementary readings (3)
Week 6, Module IILong-context systems: sparse attention and hierarchical KV memory
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
Yuan et al., ACL 2025
NSA combines coarse-grained token compression with fine-grained token selection, and its kernels are designed around arithmetic intensity on modern GPUs. Because it is trainable end to end, it lowers pretraining cost as well as inference cost. A model pretrained with NSA matches or exceeds full attention on general, long-context, and reasoning benchmarks, and at 64K tokens it is 11.6× faster in decoding, 9× in the forward pass, and 6× in the backward pass.
Strata: Hierarchical Context Caching for Long Context Language Model Serving
Xie et al., OSDI 2026
Long-context serving keeps KV caches across GPU HBM, CPU memory, and SSD, and simple designs become I/O-bound: fragmented layouts produce small transfers, and prefill waits while the cache loads. Strata decouples the GPU and host KV layouts so transfers are large, and its scheduler accounts for cache-loading latency and concurrent misses on the same context. Deployed in SGLang, it improves throughput by up to 5× over vLLM with LMCache and 3.75× over TensorRT-LLM.
Supplementary readings (3)
Week 7, Module IILow precision as a systems problem: INT4, KV4, and FP4
QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
Lin et al., MLSys 2025
INT4 methods speed up small-batch inference but lose that gain in large-batch cloud serving, where dequantization on CUDA cores adds 20% to 90% runtime overhead. QServe's QoQ algorithm uses 4-bit weights, 8-bit activations, and a 4-bit KV cache, with progressive quantization and SmoothAttention to preserve accuracy. It raises maximum serving throughput over TensorRT-LLM by 1.2× to 3.5×, depending on the model and GPU.
Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization
Egiazarian et al., ICLR 2026
This is the first broad study of the MXFP4 and NVFP4 formats for post-training quantization. NVFP4's small groups cancel the usual outlier mitigation, and MXFP4's power-of-two scales add large error. The authors propose MR-GPTQ, a GPTQ variant with block-wise Hadamard rotations, together with GPU kernels, and reach up to 2.2× end-to-end speedup over FP16 on B200 and 4× on RTX 5090. They conclude that FP4 is not an automatic upgrade over INT4.
Supplementary readings (3)
Week 8, Module IIBreaking the autoregressive bottleneck: speculation and parallel generation
Speculative Decoding: Performance or Illusion?
Liu et al., MLSys 2026
The authors run the first systematic study of speculative decoding inside a production engine, vLLM, covering n-gram, EAGLE and EAGLE-3, draft models, and multi-token prediction across workloads, model sizes, and batch sizes. Verification by the target model dominates execution time, and acceptance length varies widely across token positions, requests, and datasets. Measured speedups fall well short of the theoretical upper bound the authors derive.
Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
Wang et al., ICLR 2026
Diffusion LLMs can decode several tokens per step, but no open model had been faster than an autoregressive model of the same size. Discrete diffusion forcing (D2F) makes generation block-wise autoregressive so the KV cache can be reused, and lets later blocks start before earlier ones finish. D2F models decode more than 2.5× faster than LLaMA3 and Qwen2.5 on GSM8K, and more than 50× faster than vanilla diffusion LLMs at comparable quality.
Supplementary readings (4)
Week 9, Module IIIRL post-training at scale: reasoning, rollouts, and systems bottlenecks
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI, Nature 2025
DeepSeek-R1 shows that reasoning behaviors such as self-reflection and verification can emerge from reinforcement learning with verifiable rewards, with no human-labeled reasoning traces. On mathematics, coding competitions, and STEM tasks, the trained model beats counterparts trained with supervised learning on human demonstrations. Its reasoning patterns are then distilled into smaller models.
RollPacker: Taming Long-Tail Rollouts for RL Post-Training with Tail Batching
Gao et al., NSDI 2026
In synchronous RL post-training, a few very long responses leave most GPUs idle during each rollout step. RollPacker's tail batching moves prompts that produce long responses into a few dedicated long rounds, so most rounds stay short and balanced while training stays on-policy. It also adapts parallelism for rollout, schedules reward computation, and streams training. On up to 128 H800 GPUs it speeds up end-to-end training by 2.03× to 2.56× over veRL.
Supplementary readings (3)
Week 10, Module IIITest-time compute: adaptive inference budgets and reasoning efficiency
Scaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Model Parameters
Snell et al., ICLR 2025
The paper asks how much a fixed amount of inference-time compute can improve a model's answer, comparing search against process-based verifiers with revising the response at test time. The best method depends on prompt difficulty, and allocating compute per prompt is more than 4× more efficient than best-of-N. In FLOPs-matched tests, a smaller model with extra test-time compute outperforms a 14× larger model on problems where the small model has some success.
Plan and Budget: Effective and Efficient Test-Time Scaling on Large Language Model Reasoning
Lin et al., ICLR 2026
Reasoning models often overthink simple questions, and fixed token budgets make them underthink hard ones. Plan-and-Budget splits a query into sub-questions and gives each a token budget based on its estimated difficulty, guided by a budget-allocation model and an E³ metric that combines correctness and cost. It gives up to 70% accuracy gains and 39% fewer tokens, and lets DS-Qwen-32B match the efficiency of DS-LLaMA-70B without retraining.
Supplementary readings (2)
Week 11, Module IVAgentic systems at scale: long-horizon evaluation and workflow orchestration
AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts
Li et al., ACL 2026
AgencyBench has 138 tasks in 32 real-world scenarios that cover six agentic capabilities. A task averages 90 tool calls, 1 million tokens, and hours of execution. A simulated user gives iterative feedback and a Docker sandbox scores deliverables against rubrics, so evaluation runs without a human in the loop. Closed-source models score 48.4% against 32.1% for open-source models.
Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platforms
Chaudhry et al., OSDI 2026
Agent frameworks expose workflows as opaque sequences of model and tool calls, which ties agent logic to specific models and hardware. Murakkab separates the workflow specification from its execution configuration, and a profile-guided optimizer with an adaptive runtime maps each component to models and hardware under user-defined SLOs. Compared with current systems it reduces GPU usage by up to 2.8×, energy by 3.7×, and cost by 4.3×.
Supplementary readings (4)
Week 12, Module IVMultimodal and composite models: long context, modality pipelines, and serving
Qwen3-VL Technical Report
Bai et al., 2025
Qwen3-VL is a family of vision-language models with a native 256K-token context that interleaves text, images, and video. It comes in dense (2B to 32B) and MoE (30B-A3B and 235B-A22B) sizes for different latency and quality targets. The main architectural changes are interleaved-MRoPE for spatial and temporal positions, DeepStack fusion of multi-level ViT features, and text timestamps for video.
M*: A Modular, Extensible, Serving System for Multimodal Models
Jha et al., 2026
Composite models combine vision encoders, language backbones, diffusion heads, audio codecs, and action generators, while serving frameworks assume one model structure. M* represents a model as a dataflow graph, the Walk Graph, and serves each request as a traversal over it, with flexible placement of components on a cluster. It lowers text-to-image latency by 20% on average compared with vLLM-Omni and gives up to 2.7× higher text-to-speech throughput on Qwen3-Omni.
Supplementary readings (3)
Week 13, Module VHardware-software co-design: wafer scale, SRAM pipelines, dataflow, and memory
WaferLLM: Large Language Model Inference at Wafer Scale
He et al., OSDI 2025
WaferLLM is the first LLM inference system for wafer-scale accelerators, which place hundreds of thousands of cores and tens of GB of distributed on-chip memory in a mesh. A hardware model called PLMR guides its parallelism and its MeshGEMM and MeshGEMV kernels. On a Cerebras WSE-2 it runs GEMV 606× faster than an A100 and full inference 10× to 20× faster than A100 clusters running SGLang and vLLM.
SHIP: SRAM-Based Huge Inference Pipelines for Fast LLM Serving
Bitar et al., MLSys 2026
Groq describes its first-generation SRAM-based inference deployment, which serves hundreds of billions of tokens a day. Keeping weights in on-chip SRAM avoids the HBM bandwidth limit of GPU decoding. The paper covers the synchronous, low-diameter interconnect across thousands of chips, serving with limited memory capacity, and pipeline design under changing prefill-to-decode ratios and context lengths.
Supplementary readings (4)
Week 14Final project presentations
Groups present their survey and experiment projects on the last day of classes.

A survey backed by an experiment

Groups write a survey on a focused topic from the course and run a real experiment to support it. Each proposal states a concrete scaling claim or trade-off to test, for example throughput against batch size, reasoning accuracy against token budget, context length against KV-cache footprint, agent success against tool-call budget, or accuracy against precision.

Proposals are due in Weeks 3 to 4, and groups present in Week 14, the last week of classes. A project can earn up to 10 bonus points for publishable work or a meaningful open-source contribution, and strong projects can be connected with industry mentors.

More on teaching at Columbia

The Scaling LLMs seminar builds on High-Performance Machine Learning, the graduate course on GPU execution, distributed training, compression, and LLM serving. The teaching section on the homepage covers both courses.