SGLang

by LMSYS / sgl-project

A very fast LLM serving engine, used in production for large open models.

37 k stars GitHub Apache-2.0 Free when localLocalOpen sourceOpen weights

View on GitHub →

What it does

RadixAttention: cache reuse across requests
Strong performance on MoE models such as DeepSeek
OpenAI-compatible API and an official Docker image

Install it

Requirements: Linux with an NVIDIA or AMD GPU, Docker or Python.

Docker image

docker pull lmsysorg/sglang:latest

Or with pip (uv)

uv pip install --prerelease=allow sglang

Documentation

https://docs.sglang.io

The prompt for your agent

Paste it into your coding agent: it checks the requirements and installs it for you.

Claude CodeCodexCursorGemini CLIopencode

Prompt

Install SGLang (https://docs.sglang.io) on my machine.
1. Check the requirements: Linux with an NVIDIA or AMD GPU, Docker or Python. If anything is missing, tell me before installing anything.
2. Follow the official method:
- Docker image: docker pull lmsysorg/sglang:latest
- Or with pip (uv): uv pip install --prerelease=allow sglang
- see https://docs.sglang.io
3. Check that it works, then explain how to launch it and where to start.

Always review what your agent is about to run, especially install commands.