OfflinAI home
Request early access
Recipe

A private ChatGPT for your office

OfflinAI Server runs the AI models; Open WebUI, an open-source chat page, gives people accounts, chat history and questions about their documents. Both run on your own computer, and nothing is sent to an outside AI service. About 15 minutes, plus the downloads. Works on the Free plan.

Before you start

What you need

  • A 64-bit Intel or AMD (x86-64) computer with Windows, Linux or macOS, and Docker (Docker Desktop on Windows and macOS). ARM computers, including Apple silicon, aren't supported yet.
  • 8 GB of memory or more. A graphics card isn't needed for small models; it makes replies faster.
  • About 15 GB of free disk space, and an internet connection while you set it up. After that it runs without one.

This sets up one computer: only the computer running Docker can open the chat page. Serving colleagues on their own computers needs a Pro licence or above.

compose.yamldownload
# Save as compose.yaml in an empty folder
services:
  offlinai-server:
    image: offlinai/offlinai-server:0.1.4
    command: ["serve", "--network", "0.0.0.0:12443",
              "--allow-host", "offlinai-server"]
    ports: ["127.0.0.1:12436:12436"]
    volumes: [oas-data:/data]
    restart: unless-stopped
  open-webui:
    image: ghcr.io/open-webui/open-webui:v0.12.3
    depends_on:
      offlinai-server: { condition: service_healthy }
    ports: ["127.0.0.1:3000:8080"]
    volumes: [open-webui-data:/app/backend/data]
    # + the settings in the downloaded file: models and
    #   document indexing from OfflinAI Server, offline
    #   mode, no telemetry, sign-ups wait for approval
volumes: { oas-data: {}, open-webui-data: {} }
Steps

Set it up

  1. Get the file

    Download compose.yaml into an empty folder and open a terminal (PowerShell on Windows) there.

  2. Start OfflinAI Server

    docker compose up -d offlinai-server. When http://127.0.0.1:12436/readyz answers {"status":"ready"}, it's running.

  3. Set the console password

    Open http://127.0.0.1:12436/console/ on the same computer and choose an administrator password.

  4. Download two models

    In Models: a chat model (for a first try, Qwen2.5 1.5B Instruct, about 1 GB; Llama 3.2 3B answers better if you have the memory) and the embedding model nomic-embed-text (about 270 MB), which lets Open WebUI find the right part of a document.

  5. Create an API key

    In API keys, create a key named open-webui and copy it: it's shown only once. Open WebUI runs in its own container, so it needs a key; its usage then shows on the console's Usage page.

  6. Save the key

    Next to compose.yaml, create a plain-text file named .env with OAS_API_KEY= followed by your key, and WEBUI_SECRET_KEY= followed by any long random string. Keep it private.

  7. Start Open WebUI

    docker compose up -d. The first start takes a minute or two.

  8. Create the Open WebUI administrator

    Open http://127.0.0.1:3000. The first account you create is Open WebUI's administrator (separate from the server's console password). Later sign-ups wait for your approval.

  9. Chat, and ask about documents

    Choose the chat model at the top of the page and ask a question. To ask about a document, click +, upload it, and ask. The embedding model also appears in the model list; hide it in Admin Panel → Settings → Models.

Next

Serving the whole office

With a Pro licence or above: activate it on the console's Licence page, put Open WebUI behind HTTPS on your network (a reverse proxy with a certificate for a name such as chat.office.lan), and change WEBUI_URL and CORS_ALLOW_ORIGIN in compose.yaml to that address. Keep port 12436 on 127.0.0.1. The administrator guide, included with every licence, covers certificates, single sign-on and graphics cards.

Questions about your setup? Tell us about it and a person will reply by email.

Open WebUI's licence

Leave its branding in place

Open WebUI isn't made by OfflinAI. Its licence (based on BSD 3-Clause) says its name and logo may not be removed or changed when more than 50 people use an installation in any rolling 30 days, unless you have Open WebUI's written permission or enterprise licence. The simplest way to comply is to leave its branding as it is. Read the licence before you deploy, and again when you upgrade.

Questions

When something goes wrong

Still stuck? Ask us.

Open WebUI shows no models

Check the key in .env, that it isn't revoked on the console's API keys page, and that a model is downloaded. Then run docker compose up -d again.

Uploading a document fails

Check that the embedding model is downloaded. If you chose a different one, add its name to .env, for example OAS_EMBEDDING_MODEL=all-minilm:latest.

Creating a key says the plan limit is reached

The Free plan allows two API keys. Revoke one you no longer use, or activate a licence.

Replies are slow

Use a smaller model, or a graphics card. The first reply after a pause is slower, because the model is loaded into memory again.

Does Open WebUI send anything to the internet?

This file runs it in offline mode with update checks, usage statistics and community sharing turned off, and it uses OfflinAI Server for both chat and document indexing. Web search, image generation and other features that call outside services stay off unless you turn them on.