SGLang Info
Overview of SGLang, CI infrastructure, and Docker images
Stable Docker Image
Official SGLang release images for production use.
lmsysorg/sglang
Docker Hub
docker pull lmsysorg/sglang:latest
ROCm Nightly Image
AMD ROCm nightly builds for MI300X/MI325X/MI355X development.
rocm/sgl-dev
Docker Hub
docker pull rocm/sgl-dev:latest
What is SGLang?
SGLang is a fast serving framework for large language models and vision-language models. It makes your model fast and easy to use with a flexible frontend language and powerful backend runtime.
For an in-depth introduction with setup instructions and examples on AMD Instinct GPUs, see the ROCm blog post: SGLang on AMD Instinct GPUs.
Key Capabilities
- High-throughput LLM/VLM inference serving
- Supports 100+ model architectures: dense LLMs, MoE, VLMs, diffusion models
- Multi-GPU: tensor parallelism (TP), expert parallelism (EP), data parallelism (DP)
- FP8/INT4 quantization, continuous batching, RadixAttention (prefix caching)
- Multi-platform: NVIDIA, AMD (MI300X/MI325X/MI355X), Ascend NPU
- Community cookbook: cookbook.sglang.io
What is SGLang CI?
SGLang's upstream CI (sgl-project/sglang) runs automated tests on every PR and nightly to validate correctness and performance across GPU platforms.
AMD CI Infrastructure
| Self-hosted runners |
linux-mi325-gpu-8 (MI300X/MI325X)linux-mi35x-gpu-8 (MI355X)
|
| ROCm versions | ROCm 7.0 (default) and ROCm 7.2 |
| Execution | Docker-based: each job spins up a ROCm container, installs dependencies, runs tests |
| Test registration | Tests live in test/registered/amd/ and register via register_amd_ci() |
What is Nightly CI?
Nightly CI runs every night (not on every PR) to test things that are too expensive or slow for per-PR testing:
| Type | What it tests | Example |
|---|---|---|
| Accuracy | Run models end-to-end, check output quality against benchmarks | MiniMax-M2.5, DeepSeek-V3.2, GLM-5 |
| Performance | Measure throughput/latency, detect regressions | DeepSeek-V3.2 MTP decode speed |
| Large models | 8-GPU models too big for per-PR runners | Kimi-K2.5 (TP=8), Qwen3-235B |
nightly-test-amd.yml (ROCm 7.0) &
nightly-test-amd-rocm720.yml (ROCm 7.2).
Each model gets up to 4 nightly jobs (MI30x + MI35x, each on ROCm 7.0 + 7.2).
SGLang Cookbook
The SGLang Cookbook is a community-maintained site with practical guides for deploying SGLang with various models on different hardware. Includes AMD-specific deployment guides.