LocalAI

by Ettore Di Giacinto (mudler)

A local, open-source alternative to the OpenAI API: text, images, audio, all on your machine.

49.5 k stars GitHub MIT Free when localLocalOpen sourceOpen weights

View on GitHub →

What it does

OpenAI-compatible API, a drop-in replacement for your apps
Runs on CPU, or NVIDIA, AMD, Intel and Vulkan GPUs
Loads models from a gallery, Hugging Face or Ollama

Install it

Requirements: Docker (the easiest way).

Run (CPU)

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest

Run with an NVIDIA GPU (CUDA 12)

docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

The prompt for your agent

Paste it into your coding agent: it checks the requirements and installs it for you.

Claude CodeCodexCursorGemini CLIopencode

Prompt

Install LocalAI (https://localai.io) on my machine.
1. Check the requirements: Docker (the easiest way). If anything is missing, tell me before installing anything.
2. Follow the official method:
- Run (CPU): docker run -ti --name local-ai -p 8080:8080 localai/localai:latest
- Run with an NVIDIA GPU (CUDA 12): docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12
3. Check that it works, then explain how to launch it and where to start.

Always review what your agent is about to run, especially install commands.