AnythingLLM is a full-stack application for building private, document-grounded AI assistants. Upload any document — PDF, Word, text, Markdown, GitHub repo — and chat with it using a local LLM. No data leaves your server. No API costs per query. No subscription.

What Is RAG?

RAG (Retrieval-Augmented Generation) takes your query, retrieves relevant chunks from your documents, injects them into the prompt as context, and the model answers using that context. This lets you query documents the model never saw — your codebase, contracts, research papers.

What You Get

  • Document ingestion: PDF, DOCX, TXT, Markdown, CSV, Google Drive, GitHub, web pages
  • Workspace isolation: Separate workspaces for different projects
  • Built-in LanceDB vector database (no separate DB setup)
  • LLM backends: Ollama, LocalAI, OpenAI, Azure, Anthropic
  • Multi-user with shared or private workspaces
  • Citation tracking: See which chunks the answer came from
  • Vision support with multimodal models (llava)

Installation (Docker)

docker pull mintplexlabs/anythingllm:latest

docker run -d \
  --name anythingllm \
  -p 3000:3000 \
  -v $HOME/anythingllm:/app/server/storage \
  -v $HOME/anythingllm/.env:/app/server/.env \
  --network host \
  mintplexlabs/anythingllm:latest

Access at http://your-server:3000

Configuration

# In $HOME/anythingllm/.env
LLM_PROVIDER=ollama
OLLAMA_BASE_URL=http://127.0.0.1:11434
OLLAMA_MODEL=llama3
OLLAMA_EMBEDDING_MODEL=nomic-embed-text

ollama serve &
ollama pull llama3
ollama pull nomic-embed-text

First Workspace

  1. Create an admin account
  2. Click + New Workspace
  3. Upload documents (PDF, DOCX, TXT, MD, CSV)
  4. Wait for processing (chunking + embedding)
  5. Start chatting with your documents

Click any answer to see which document chunks were used — full citation transparency.

Vision Models

ollama pull llava

Set in workspace: Vision Model: llava. Now upload images and ask questions about them.

API Access

curl -X POST http://localhost:3000/api/workspace/chat \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"workspaceSlug": "my-workspace", "message": "Summarise this contract"}'

Troubleshooting

No embeddings found: Run ollama pull nomic-embed-text and re-index documents.

Slow processing: Use a GPU for faster embedding generation.

Context exceeded: Reduce MAX_CHUNKS in workspace settings.