Data stays home
Prompts, documents and answers never leave your machines. Run it on a laptop, a GPU box or a cluster — or with no internet at all.
The product
OfflinAI Server
Your own private AI server: chat, vision, speech and embedding models on your own hardware, behind the API your software already speaks. Nothing leaves your network.
- OpenAI-compatible API
- Windows and Linux
- Works air-gapped
- Every plan runs any model
Point any OpenAI SDK at your serverPython
from openai import OpenAI
client = OpenAI(
base_url="https://ai.example.internal:12443/v1",
api_key="ofai_sk_…",
)
reply = client.chat.completions.create(
model="ollama/qwen2.5/7b",
messages=[{"role": "user",
"content": "Summarise this contract."}],
)
print(reply.choices[0].message.content)
Overview
The AI server you run yourself
One program that serves every kind of model through the API your tools already use, with the controls a business needs — on a laptop, a GPU box, a cluster, or a network with no internet at all.
Drop-in compatible
OpenAI's API by default, plus Ollama's and Anthropic's. Existing tools, SDKs and apps work by changing one URL.
Any model, any GPU
A built-in library of open-weight models, any GGUF file and Ollama models; NVIDIA GPUs through CUDA, any GPU through Vulkan, or plain CPU — one worker per GPU.
Ready for IT
API keys and quotas, single sign-on, a signed audit log, metrics and traces, a Windows service and Linux packages.
Models
Every kind of model, one server
Each capability runs in its own supervised worker process: a crash restarts the worker, not the server, and requests in flight are accounted for exactly.
Chat and agents
Streaming chat with tool calls, JSON and JSON-schema output, log-probabilities and several choices per request; the Responses API with stored conversations and background runs.
Vision
Send photos, receipts and screenshots with your prompt. Images are decoded on the server itself, and the server never fetches a URL on a caller's behalf.
Embeddings and reranking
Vectors for search and retrieval, and rerankers that put the right documents first — the building blocks of private document search.
Speech to text
Transcription and translation with timestamps, streamed results, and live transcription over a WebSocket with voice-activity detection and live captions.
Text to speech
Natural voices streamed as they are spoken, in WAV, FLAC or raw PCM.
Moderation
Guard models (Llama Guard 3, IBM Granite Guardian) score text — and a vision model screens images — in OpenAI's moderation format.
Files and batches
OpenAI's Batch API for large jobs: durable across restarts, every line answered once.
Scale out
Several GPUs in one machine, or other machines joined as remote workers over mutual TLS. Requests go where the model is already loaded.
A model library
About a hundred open-weight models in the console, each with its size, licence and source. Filter by what fits your GPU and download with one click — it works offline.
Which models? Any GGUF model, the models of an existing Ollama store, and the server's built-in library of about a hundred open-weight models, which works offline, with a search of Hugging Face for the rest — chat and vision models, embedding models, rerankers, guard models, Whisper speech models and text-to-speech models. Every plan runs any model.
API
Open standards first
OfflinAI Server speaks OpenAI's API as its default contract, so OpenAI's SDKs and the tools built on them work by pointing them at your server. It also answers Ollama's API and Anthropic's Messages API, exports traces over OpenTelemetry, and joins the caller's W3C trace.
/v1/chat/completions/v1/responses/v1/embeddings/v1/audio/transcriptions/v1/audio/speech/v1/moderations/v1/rerank/v1/batches/api/*Ollama/v1/messagesAnthropic
Transcribe a meeting recordingcurl
curl https://ai.example.internal:12443/v1/audio/transcriptions \
-H "Authorization: Bearer $OAS_KEY" \
-F file=@meeting.m4a -F model=whisper-1 \
-F response_format=verbose_jsonEmbed documents for searchcurl
curl https://ai.example.internal:12443/v1/embeddings \
-H "Authorization: Bearer $OAS_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"nomic-embed-text","input":["first","second"]}'
For IT
Built for the people who run it
A web console in 17 languages, and every control your security team will ask about.
Keys, limits and budgets
API keys scoped to endpoints and models, with request and token limits, daily and monthly quotas, network restrictions and budgets per project. Exact usage from the engines' own counts.
Organisations and single sign-on
Organisations, projects and users with roles; sign-in with Microsoft Entra ID, Google, Okta or any OpenID provider.
Tamper-evident audit log
Every administrative action in a signed, hash-chained log you can verify, forwarded as it happens to your SIEM over syslog or HTTPS.
Content filters
Mask or block personal data — emails, phone numbers, card numbers, IBANs and your own patterns — in requests and replies, and have a guard model read them.
Encrypted by default
HTTPS for other machines with a built-in certificate authority, your own certificates or automatic ones (ACME), and client certificates for mutual TLS.
Observable
Prometheus metrics, OpenTelemetry request traces, daily log files and a live console view of what every worker is doing.
Air-gapped licensing
Activate online, or carry a licence request to a connected machine and back: the server never needs to call home.
AI output marked
Generated speech carries metadata that says it is AI-generated, and every generated answer says so in a response header — as the EU AI Act asks.
Deploy
Deploy it your way
A 64-bit Intel or AMD (x86-64) Windows or Linux machine is all it needs. Memory decides which models fit: small models run on a laptop, large ones want a GPU with plenty of memory, or several.
Windows
A Windows service that starts with the machine, or a tray app for one person.
Linux
Debian and RPM packages with hardened systemd units.
Containers
The image offlinai/offlinai-server on Docker Hub.
Kubernetes
A Helm chart for Kubernetes clusters, and private registries welcome.
Microsoft Store
Coming soon to the Microsoft Store
OfflinAI Server is coming to the Microsoft Store as a free download for Windows.
- Free: it serves the apps on this PC, such as the OfflinAI Lite apps, with the models you choose.
- Pro, from offlinai.com: adds serving other machines on your network over HTTPS, more engine workers, more API keys, projects and users, longer usage history and batch requests.
It is in Microsoft Store certification now. We will link it here when it is published.
Coming soon to the Microsoft Store
38 free Lite apps use it
Notepad, Docs, Sheets, Slides, Chat, Translate, Dictation, Mail, Tasks and more: small Windows apps that get their AI from OfflinAI Server on the same PC.
Plans
Every plan runs any model
Plans differ in what the server does around the models: serving other machines, keys and users, single sign-on, policies and audit. Prices are per server, not per seat.
Documentation
The full guide ships with the server
Every licensed server carries the complete administrator and developer guide for its own version, in the web console under Guide, in the console's language.
Evaluating for your organisation? Request early access and we will share the guide with your team.
What the guide covers
- Installing on Windows, Linux, containers and Kubernetes
- Licensing, including air-gapped activation
- Models, GPUs and remote workers
- Every API with examples, and the differences from OpenAI's service
- Keys, single sign-on, policies, content filters and the audit log
- Operations, monitoring, backup, settings and troubleshooting
Try it
Try it in a container
The Free plan runs from the image on Docker Hub. With Docker on Windows, macOS or Linux, it serves programs on that computer.
Image: offlinai/offlinai-server. Serving other machines over HTTPS (port 12443) needs a licence that includes it. Using it means accepting the OfflinAI Server Licence.
Want a chat page for your office on top? Follow the OfflinAI Server + Open WebUI recipe: a private ChatGPT with document questions in about 15 minutes.
Start the Free planshell
# Start the server (models and settings live in the volume)
docker run -d --name oas -p 127.0.0.1:12436:12436 \
-v oas-data:/data offlinai/offlinai-server
# Set the console's password in your browser
http://127.0.0.1:12436/console/
# Any OpenAI client: base URL http://127.0.0.1:12436/v1Does it need an internet connection?
No. Once your models are downloaded the server runs entirely offline. Licences can be activated online, or offline for air-gapped sites: you carry a small request file to any connected machine and bring the signed licence back.
Which models can I run?
Any GGUF model, the models of an existing Ollama store, and the server's built-in library of about a hundred open-weight models, which works offline, with a search of Hugging Face for the rest — chat and vision models, embedding models, rerankers, guard models, Whisper speech models and text-to-speech models. Every plan runs any model.
What hardware do I need?
A 64-bit Intel or AMD (x86-64) Windows or Linux machine; Windows on ARM and ARM Linux aren't supported yet. NVIDIA GPUs run through CUDA and other GPUs through Vulkan; without a GPU the server runs on the CPU. Memory decides which models fit: small models run on a laptop, large ones want a GPU with plenty of memory or several GPUs.
Will my existing OpenAI code work?
Yes — set the SDK's base URL to your server and use one of its API keys. The differences from OpenAI's own service are listed in the documentation: for example, speech comes back as WAV, FLAC or PCM rather than MP3.
What does the server send to OfflinAI?
Never your prompts, documents or outputs. The Free plan sends nothing. A paid plan activates and refreshes its licence with our licence service, sending the licence key, a pseudonymous identifier of the machine, its host name, its operating system and the server's version; a server without internet access uses an offline licence file instead.
Is macOS supported?
In Docker Desktop, yes: the Free plan's container runs on a Mac and serves programs on it, on the CPU (containers on a Mac can't use its GPU; on Apple silicon Docker runs the image through emulation, so it is slower). There is no native macOS version yet: for a GPU, run the server on Windows or Linux.
Early access
Run AI where your data already lives
OfflinAI Server is in early access. Tell us where you'd like to run it and we'll be in touch with a licence and the packages for your platform.
- A licence and the packages for your platform
- A free 20-minute call to size your hardware
- Founding-customer pricing: 50% off your first year
- A reply from a person, not a sales sequence
Tell us about your setup
Where would you like to run OfflinAI Server, and for what? We will be in touch by email.